Projects/docs/ReviewApplication.md

## 🧩 1. Service Layer (Business Logic)

These classes orchestrate repositories, apply business rules, and are called by route handlers. Many are referenced in your route files (e.g., `ArticleRoutes`, `AuthRoutes`, `MessagingRoutes`) but are missing or only stubs.

| Service | Methods to Implement |
|---------|----------------------|
| `UserService` | register, login, findUserByUsername, updateProfile, changeEmail, changePassword, deactivate, follow, unfollow, getFollowers, getFollowing, getProfile (public/private), updateSecurityQuestion, updateNotificationSettings, getNotificationSettings, saveAvatar, getBookmarks, toggleBookmark, getRecentActivity, updateLastActive, isPrivate, etc. |
| `ArticleService` | create, update, delete, getArticle, getArticleDTO, search (with filters & pagination), getAllArticles, getArticlesByAuthor, toggleLike, isLikedBy, addComment, deleteComment, getComments, generateCitation, getAnalytics (user), getArticleAnalytics (views over time), incrementViewCount, getTrendingArticles, getFeaturedArticles, getGraphRecommendations (co‑citation, bibliographic, PageRank), publishDraft, batchDelete, batchPublish. |
| `MessageService` | sendMessage, getConversation, getMessagesBefore, getMessagesAfter, markAllRead, editMessage, deleteMessage, forwardMessage, getLastReadId, updateLastRead, recordTyping, isUserTyping, archiveConversation, unarchiveConversation, getArchivedConversations, sendEphemeral, scheduleMessage, getScheduledMessages, cancelScheduledMessage, getThreadReplies, replyToMessage, getEditHistory, getPrivateReactions, togglePrivateReaction. |
| `GroupService` | createGroup, getGroup, listUserGroups, updateGroup, deleteGroup, addMember, removeMember, promoteToAdmin, demoteFromAdmin, leaveGroup, generateInviteToken, joinByInviteToken, sendMessage, editMessage, deleteMessage, getMessages, getMessagesBefore, getMessagesSince, pinMessage, unpinMessage, getPinnedMessages, muteGroup, unmuteGroup, getMutedGroups, searchMessages, createPoll, getPoll, votePoll, closePoll, reopenPoll, deletePoll, getPollResults, getReactions, toggleReaction. |
| `NotificationService` | createNotification, getUserNotifications, markAsRead, markAllAsRead, deleteNotification, deleteAllForUser, getUnreadCount, sendRealTimeNotification (via WebSocket), getNotificationById. |
| `AnalyticsService` | getUserStats (total articles, likes, comments, views, followers, etc.), getUserViewsOverTime, getTopArticlesForUser, getAdminStats (platform), getPlatformActivity, getTopContributors, getArticleAnalytics (views, likes, comments per day), getDailyViews, getReferrerStats, getDemographics, getCitationMetrics, setViewThresholdAlert, getActiveAlerts, deleteAlert, getTrendingArticles, getPopularTags. |
| `GapAnalysisService` | getAllNodes, getNode, addNode, updateNode, deleteNode, searchNodes, getNodesByType, getAllEdges, getEdge, addEdge, updateEdge, deleteEdge, getEdgesForNode, getEdgesByType, getNodeDegrees, analyzeGap (AI), generateHypothesis, importGraph, exportGraph. |
| `GrantService` | getUpcomingGrants, getGrantsForUser, addGrant, updateGrant, deleteGrant, generateDraft (AI), analyzeRFP. |
| `ProjectService` | createProject, getUserProjects, getProjectsUserIsMember, getProject, updateProject, deleteProject, addMember, removeMember, setMemberRole, getMemberRole, getMemberJoinedAt, addTask, getTasksForProject, getTask, updateTask, moveTask, deleteTask, addOutput, removeOutput. |
| `DataHubService` | getUserResearchObjects, getAllResearchObjects, getResearchObject, addResearchObject, updateResearchObject, deleteResearchObject, incrementResearchObjectView, incrementResearchObjectDownload, getUserProtocols, getAllProtocols, getProtocol, addProtocol, updateProtocol, deleteProtocol, forkProtocol, getProtocolSteps, addProtocolStep, updateProtocolStep, deleteProtocolStep, getUserPreregistrations, getPreregistration, addPreregistration, updatePreregistration, updatePreregistrationSection, submitPreregistration, deletePreregistration. |
| `PollService` | createPoll, getPoll, updatePoll, deletePoll, vote, getUserVote, getPollResults, getPollAnalytics, exportResultsToCsv, getPollsByCreator, getPollTemplates, instantiateTemplate, schedulePoll, setPollPrivacy, searchPolls, deleteExpiredPolls, hasAnyVotes, isUserInGroup, isUserInProject. |
| `EventService` | listEvents, getUpcomingEvents, getUserEvents, getEvent, createEvent, updateEvent, deleteEvent, incrementViewCount, setAttendeeStatus, getAttendees, addReminder, getRemindersForUser, deleteReminder, createRecurringEvents, exportUserEventsToIcal, searchEvents. |
| `LabInventoryService` | getAllChemicals, getChemical, addChemical, updateChemical, deleteChemical, getLowStockCount, getExpiringSoonCount, getExpiredCount, getUnacknowledgedAlertsCount, getAllLocations, getSupplierCount, getTotalScanCount, getRecentlyAdded, getTopExpiring, getCountByLocation, getCountByType, getIncompatibilities, areCompatible, checkAndCreateAlerts, acknowledgeAlert, acknowledgeAllAlertsForChemical, createAlert, getAlert, deleteAlert, getChemicalByBarcode, getChemicalByCasNumber, incrementScanCount. |
| `PeerReviewService` | submitPaper, getSubmissionById, getOpenSubmissions, getSubmissionsByStatus, getSubmissionsAssignedToReviewer, assignReviewer, submitReview, getReviewById, getReviewsForSubmission, getReviewsByReviewer, getReviewerCredential, saveReviewerCredential, deleteReviewerCredential, getAllSubmissions, getAllReviews, deleteSubmission. |
| `WorkspaceService` | getUserPanels, getPanelById, createPanel, updatePanel, renamePanel, deletePanel, duplicatePanel, reorderPanels, exportWorkspaceToJson, importWorkspaceFromJson, clearAllPanels, getBuiltInPresets, applyPreset, createShareToken, getSharedPanel, getActivePanel, setActivePanel, createBackup, restoreBackup, listBackups, deleteBackup. |
| `VoiceSettingsService` | getSettings, updateSettings, resetToDefault, getAllProfiles, getProfile, saveProfile, deleteProfile, activateProfile, getLanguageOverrides, setLanguageOverrides, getAvailableTtsVoices, generateTestAudio, listWakeWords, addWakeWord, removeWakeWord, getCustomGrammar, setCustomGrammar, exportAllSettings, importAllSettings. |
| `RagService` | chatSimple, chat (RAG), getAvailableModels, getSuggestedQuestions, indexDocument, getIndexingStatus, getUserDocuments, deleteDocument, getConversationHistory, clearConversationHistory, deleteMessageFromHistory, predictCommands, clearAllHistory, reindexAllDocuments, getRagStats, isLLMHealthy, getEmbeddingModelInfo. |
| `PasswordResetService` | getSecurityQuestion, resetPassword. |
| `TwoFactorService` | generateSecret, getQrCode, verifyCode, enableTwoFactor, disableTwoFactor, generateBackupCodes, getBackupCodes, regenerateBackupCodes. |
| `SearchService` | search (messages), getIndexedCount (full‑text search statistics). |
| `MediaService` | uploadMedia, getMedia, deleteMedia, updateMediaGroupMessageId. |
| `RateLimitService` | allowRequest (uses `TokenBucket`). |
| `ContentModeration` | moderate (check profanity), censor. |
| `CitationGraphService` | getCoCitationRecommendations, getBibliographicCouplingRecommendations, getPageRankRecommendations. |
| `GNNRecommendationService` | getPersonalizedFeed, getContextualRecommendations, getTrendingArticles, getSimilarArticles, getSuggestedUsers, getUsersByInterest, recordFeedback, getPersonalizedTrending, refresh, fullRetrain, getModelStats, resetModel, setEnabled. |
| `RecommendationService` | getPersonalizedFeed, getTrendingArticles, getTrendingArticlesByCategory, getTrendingArticlesByTimeRange, getSimilarArticles, getArticlesByTags, getSuggestedUsers, getSuggestedUsersByInterest, getHybridFeed, recordFeedback, getUserRecommendationHistory, refreshModel, fullRetrain, getModelStats, resetModel. |

---

## 🧩 2. Controllers / Route Handlers (Missing Endpoints)

Even though you have route skeleton files (e.g., `ArticleRoutes`, `AuthRoutes`), many endpoints are not implemented. Below are the **missing route handlers** (grouped by feature).

| Feature | Missing Endpoints | Required Service |
|---------|-------------------|------------------|
| **Recent Items** | `GET /api/recent-items` | `UserService` (or new `RecentItemService`) |
| **Unified Inbox** | `GET /api/unified-inbox` | `UnifiedInboxService` (aggregate messages, notifications, paper Q&A, broadcasts) |
| **Discovery** | `GET /api/discovery/digest`<br>`GET /api/discovery/recommendations` | `NotificationService`<br>`RecommendationService` (alias) |
| **User liked articles** | `GET /api/users/me/liked-articles` | `ArticleService` |
| **Messaging last read** | `GET /api/conversations/{username}/last-read` | `MessageService` (call `/read-status`) |
| **Secret chat keys** | `GET /api/conversations/{username}/keys` | `EncryptionKeyRepository` (already exists) |
| **Group pinned messages** | `GET /api/groups/{groupId}/pinned` | `GroupService` (alias to `/pins`) |
| **Paper Collections** | `GET /api/collections`<br>`POST /api/collections`<br>`DELETE /api/collections/{id}`<br>`POST /api/collections/{id}/papers`<br>`DELETE /api/collections/{id}/papers/{paperId}` | `CollectionService` (new) + `CollectionRepository` |
| **Reference Manager** | `GET /api/documents/{docId}/references`<br>`POST /api/documents/{docId}/references`<br>`PUT /api/documents/{docId}/references/{refId}`<br>`DELETE /api/documents/{docId}/references/{refId}`<br>`POST /api/references/connect-zotero`<br>`POST /api/references/connect-mendeley` | `ReferenceService` + `ReferenceRepository` |
| **Preprint Submission** | `POST /api/preprints/screen`<br>`POST /api/preprints/submit` | `PreprintService` + `PreprintRepository` |
| **Provenance** | `GET /api/provenance/{entityType}/{entityId}`<br>`POST /api/provenance/{entityType}/{entityId}/verify` | `ProvenanceService` + `ProvenanceEventRepository` |
| **Export** | `POST /api/ai/export`<br>`POST /api/documents/{docId}/export` | `ExportService` (calls external converter) |

Additionally, you need to **implement** the route files that are currently stubs: `ArticleRoutes`, `AuthRoutes`, `MessagingRoutes`, `UserRoutes`, `AnalyticsRoutes`, etc. – they contain placeholder `// TODO` or empty method bodies.

---

## 🧩 3. DTOs (Data Transfer Objects)

Many route handlers return or consume DTOs. You already have some (e.g., `ArticleDTO`, `CreateArticleRequest`), but many are missing.

| DTO | Purpose | Used In |
|-----|---------|---------|
| `ArticleDTO` | Return article metadata (without full body for lists) | `ArticleService` |
| `CreateArticleRequest` | Input for article creation | `ArticleService` |
| `CommentDTO` | Comment data (id, author, body, createdAt) | `ArticleService` |
| `UserProfileDTO` | Public profile information | `UserService` |
| `UserSettingsDTO` | Notification, privacy, security settings | `UserService` |
| `AnalyticsDTO` | User analytics (total views, likes, etc.) | `AnalyticsService` |
| `ViewStatDTO` | Daily views (date, count) | `AnalyticsService` |
| `MessageDTO` | Private message representation (without internal fields) | `MessageService` |
| `GroupMessageDTO` | Group message representation | `GroupService` |
| `ConversationDTO` | Private conversation summary (with, lastMessage, unread) | `MessageService` |
| `GroupDTO` | Group metadata (id, name, memberCount, lastMessage) | `GroupService` |
| `MemberDTO` | Group member (userId, username, role) | `GroupService` |
| `PollDTO` | Poll/quiz data (question, options, results) | `PollService` |
| `PollVoteDTO` | Vote submission | `PollService` |
| `ChemicalDTO` | Chemical inventory item (with NFPA, location, expiry) | `LabInventoryService` |
| `AlertDTO` | Inventory alert | `LabInventoryService` |
| `EventDTO` | Event details (title, startTime, location, host) | `EventService` |
| `ProjectDTO` | Project summary | `ProjectService` |
| `TaskDTO` | Kanban task | `ProjectService` |
| `ProtocolDTO` | Protocol with steps | `DataHubService` |
| `PreregistrationDTO` | Preregistration (template, sections) | `DataHubService` |
| `ResearchObjectDTO` | Object metadata (type, title, authors, DOI) | `DataHubService` |
| `GrantDTO` | Grant deadline (name, agency, deadline, amount) | `GrantService` |
| `RecommendationResultDTO` | Recommended article (id, title, score, tags) | `RecommendationService` |
| `ScreeningResultDTO` | Preprint screening result | `PreprintService` |
| `BackupCodesDTO` | List of backup codes | `TwoFactorService` |
| `TwoFactorSecretDTO` | Secret + QR code for 2FA setup | `TwoFactorService` |
| `VoiceSettingsDTO` | Voice assistant configuration | `VoiceSettingsService` |
| `WorkspacePanelDTO` | Panel layout (type, title, props, position) | `WorkspaceService` |
| `WorkspacePresetDTO` | Preset definition | `WorkspaceService` |

---

## 🧩 4. Utilities & Helper Classes

| Utility | Description | Required By |
|---------|-------------|-------------|
| `JsonUtils` | Jackson object mapper wrapper (you already have it in `util`) | All route handlers |
| `TokenUtil` | Extract JWT from `Authorization` header | `AuthUtils`, `AuthRoutes` |
| `SessionManager` | Create, validate, refresh sessions; manage CSRF tokens; store username → token mapping; invalidate sessions | `AuthUtils`, `AuthRoutes` |
| `WebSocketManager` | Manage active WebSocket sessions, send messages to specific users, broadcast to groups/topics | `MessagingRoutes`, `NotificationService` |
| `RateLimitService` | Use `TokenBucket` to check if request is allowed | `MessagingRoutes`, `AuthRoutes` |
| `ContentModeration` | Filter profanity using a word list or external API | `MessagingRoutes` |
| `MailService` | Send emails (verification, password reset, invitation, daily digests) | `UserService`, `PasswordResetService`, `NotificationService` |
| `FileStorageService` | Save uploaded files (avatars, manuscripts, research objects, etc.) to disk/cloud; return URL | `UserService`, `PeerReviewService`, `DataHubService` |
| `CitationGenerator` | Convert article metadata to APA/MLA/BibTeX string | `ArticleService` |
| `ExportConverter` | Convert HTML to PDF, DOCX, LaTeX (possibly using external tools like `wkhtmltopdf`, `Pandoc`) | `ExportService` |
| `PasswordEncoder` | BCrypt hashing for passwords | `UserService` (already used) |
| `TotpUtil` | Generate TOTP secret, QR code URI, verify code (wrapper around Google Authenticator) | `TwoFactorService` |
| `HtmlSanitizer` | Sanitize HTML from editor (prevent XSS) | `ArticleService`, `MessageService` |
| `MarkdownConverter` | Convert markdown to HTML (if needed) | `ArticleService` |
| `CacheManager` | Simple in‑memory cache (e.g., for frequently requested articles) | Optional – performance |
| `AsyncTaskExecutor` | Execute long‑running tasks asynchronously (e.g., document indexing, email sending) | `RagService`, `MailService` |
| `ScheduledTaskManager` | Schedule jobs (e.g., daily digest, cleanup expired tokens) | `NotificationService`, `SessionManager` |
| `ApiError` | Standard error response structure | All controllers |
| `GlobalExceptionHandler` | Catch exceptions and return appropriate HTTP status codes | All controllers |
| `ValidationUtils` | Validate email, username, password strength | `UserService`, `AuthRoutes` |

---

## 🧩 5. Security & Configuration

| Component | Description |
|-----------|-------------|
| `JwtUtil` | Generate and parse JWT tokens (claims, expiration) |
| `CsrfTokenManager` | Generate and validate CSRF tokens per session |
| `CorsConfig` | Configure CORS for frontend origin |
| `WebSecurityConfig` (Spring) | Define which endpoints are public, which require authentication |
| `PasswordEncoder` bean | BCryptPasswordEncoder |
| `AuthenticationProvider` (if using Spring Security) | Custom authentication logic |
| `ApplicationProperties` | Read environment variables (database URL, API keys, mail settings) |
| `DatabaseInitializer` | Create tables on startup (if using auto‑migration) |

---

## 🧩 6. WebSocket & SSE

| Component | Description |
|-----------|-------------|
| `WebSocketHandler` (or `@ServerEndpoint`) | Handle WebSocket connections for messaging, typing, presence, reactions. |
| `WebSocketConfig` | Register the WebSocket endpoint (`/ws`) and configure STOMP (if using) or raw WebSocket. |
| `SseEmitterManager` (SSE) | Manage SSE connections for notifications, keep‑alive, event replay. |
| `WebSocketMessage` | DTO for messages sent over WebSocket (type, payload). |

---

## 🧩 7. Database Migration & Schema

Even though you have repository classes, you must create the actual SQL tables. Provide SQL scripts (or use Flyway/Liquibase) for all entities:

- `users`
- `articles`
- `comments`
- `tags` (article_tags)
- `messages`
- `conversations`
- `groups`
- `group_members`
- `group_messages`
- `polls`, `poll_options`, `poll_votes`
- `reactions` (or store inside messages)
- `notifications`
- `chemicals`, `alerts`
- `events`, `event_attendees`, `event_reminders`
- `projects`, `project_members`, `project_tasks`, `project_outputs`
- `research_objects`
- `protocols`, `protocol_steps`
- `preregistrations`, `preregistration_sections`
- `grants`
- `graph_nodes`, `graph_edges`
- `workspace_panels`
- `voice_settings`
- `paper_collections`, `collection_papers`
- `managed_references`, `reference_groups`
- `preprints`
- `provenance_events`
- `pre_keys`, `signed_pre_keys`
- `encryption_keys` (for secret chats)

---

## 🧩 8. Additional Supporting Code

| Component | Description |
|-----------|-------------|
| **Custom Exceptions** | `ResourceNotFoundException`, `UnauthorizedException`, `BadRequestException`, `ConflictException`, etc. |
| **Validation Annotations** | `@ValidUsername`, `@ValidPassword` (custom) – optional |
| **Audit Fields** | CreatedAt, UpdatedAt automatically set (use `@PrePersist`, `@PreUpdate` in JPA) |
| **Pagination Helper** | Convert `page`/`limit` parameters to `offset`/`limit` and return paginated responses |
| **Date/Time Formatter** | Consistent ISO‑8601 formatting |
| **Health Check Endpoint** | `GET /health` (already present) |
| **Metrics Endpoint** | `GET /metrics` for Prometheus (optional) |
| **API Documentation** | Swagger/OpenAPI configuration (`springdoc-openapi`) |
| **Dockerfile** | Containerise the backend |
| **CI/CD Pipeline** | GitHub Actions / Jenkins (optional) |

---

## ✅ What You Already Have (from your files)

- **Models** – almost complete (User, Article, Comment, Message, GroupChat, GroupMessage, Poll, Chemical, etc.) – you provided many `.java` files.
- **Repositories** – you have JDBC‑based repositories (ArticleRepository, UserRepository, etc.). They are functional but may need small adjustments.
- **Route skeletons** – you have `ArticleRoutes`, `AuthRoutes`, `MessagingRoutes`, etc., but many methods are empty or only print logs. They need to be filled with actual business logic.
- **Some utilities** – `JsonUtils`, `RouteUtils`, `AuthUtils`, `TokenBucket`, etc.

---

## 🎯 Final Recommendation

Start by implementing the **services** (the longest list). Then fill in the **route handlers** (one by one, based on frontend needs). Create **DTOs** as you go. The **utilities** (Mail, FileStorage, WebSocket, etc.) can be added later but are required for full functionality.



















## 🚀 1. Infrastructure & Architecture

| Component | Technology / Implementation | Why It’s Needed |
|-----------|----------------------------|------------------|
| **API Gateway** | Spring Cloud Gateway, Kong, NGINX | Route external traffic to the correct backend service, handle SSL termination, rate limiting, authentication at the edge. |
| **Service Discovery** | Netflix Eureka, Consul, Kubernetes Service | If you split into microservices, services need to find each other dynamically. |
| **Load Balancer** | Spring Cloud LoadBalancer, HAProxy, AWS ALB | Distribute traffic across multiple instances of your backend. |
| **Reverse Proxy + CDN** | CloudFront, Cloudflare, Fastly | Serve static assets (React build) globally, cache API responses, protect against DDoS. |
| **WebSocket Server** | Spring WebSocket + STOMP (separate instance) | Handle real‑time messaging independently, scale horizontally. |
| **Database** | PostgreSQL / MySQL (with read replicas) | Primary relational store; read replicas for high‑load queries. |
| **Database Pooling** | HikariCP (default in Spring Boot) | Efficiently manage database connections. |
| **Distributed Cache** | Redis, Caffeine | Cache frequently accessed data (user profiles, session, rate limits, computed recommendations). |
| **Message Queue** | RabbitMQ, Apache Kafka | Process long‑running tasks asynchronously: sending emails, indexing documents, generating reports, AI inference. |
| **Object Storage** | AWS S3, MinIO (self‑hosted) | Store uploaded files (avatars, manuscripts, research objects, voice notes). Generate presigned URLs for secure access. |
| **Search Engine** | Elasticsearch, OpenSearch | Full‑text search across articles, messages, users, grants, chemicals – far faster than SQL `LIKE`. |
| **Distributed Locking** | Redis Redlock | Prevent race conditions (e.g., two users claiming same peer‑review assignment). |

---

## 🔐 2. Security

| Component | Technology | Purpose |
|-----------|------------|---------|
| **OAuth2 / OpenID Connect** | Spring Security OAuth2, Keycloak, Auth0 | Let users log in with Google, ORCID, institutional SSO. |
| **JWT Refresh Tokens** | Custom implementation | Short‑lived access tokens + refresh token stored in http‑only cookie for better security. |
| **CSRF Protection** | Spring Security CSRF (enabled for state‑changing endpoints) | Already partially there, but needs full enforcement. |
| **CORS Configuration** | Spring `@CrossOrigin` or global config | Restrict which origins can call your API. |
| **Rate Limiting per User / IP** | Bucket4j, Resilience4j RateLimiter | Already have `TokenBucket` – upgrade to distributed rate limiting using Redis. |
| **IP Whitelist / Blacklist** | Custom filter | Block known malicious IPs (e.g., from admin panel). |
| **HTTPS / TLS** | Let’s Encrypt, Cloudflare | Encrypt all traffic. |
| **Encryption at Rest** | Database encryption (AES‑256) + secure key management | Protect sensitive user data. |
| **Secret Management** | HashiCorp Vault, AWS Secrets Manager, Kubernetes Secrets | Store database passwords, API keys, JWT secrets securely. |
| **Audit Logs** | Custom `AuditService` + separate database table | Track who accessed what resource (GDPR requirement). |
| **Data Anonymization / Pseudonymization** | Custom service | For research data exports, remove PII. |
| **Vulnerability Scanning** | OWASP Dependency‑Check, Snyk | Scan dependencies for known CVEs. |

---

## 📈 3. Observability & Monitoring

| Component | Technology | Why Important |
|-----------|------------|---------------|
| **Structured Logging** | Logback + JSON layout, Loki (Grafana) | Make logs machine‑parseable, centralise them. |
| **Distributed Tracing** | OpenTelemetry + Jaeger / Zipkin | Trace requests across multiple services (e.g., API → DB → AI service). |
| **Metrics** | Micrometer + Prometheus + Grafana | Monitor request latency, error rates, database pool usage, queue length. |
| **Application Performance Monitoring (APM)** | New Relic, Datadog, Elastic APM | Detect slow endpoints, SQL queries, external calls. |
| **Health Checks** | Spring Boot Actuator (`/health`, `/info`, `/metrics`) | Kubernetes liveness/readiness probes, load balancer health. |
| **Alerts** | Alertmanager (Prometheus), PagerDuty, Opsgenie | Get notified when CPU spikes, error rate rises, queue backs up. |
| **Error Tracking** | Sentry, Rollbar | Capture unhandled exceptions, aggregate by type, show stack traces. |
| **Real User Monitoring (RUM)** | Sentry, Datadog RUM | See how frontend performs for real users. |
| **Synthetic Monitoring** | Grafana Cloud, Pingdom | Periodically call critical endpoints to ensure uptime. |

---

## ⚙️ 4. Messaging & Async Processing

| Component | Technology | Use Case |
|-----------|------------|----------|
| **Task Queue** | RabbitMQ, Kafka (with Spring AMQP / Kafka) | Indexing PDFs, sending emails, generating AI summaries, exporting large documents. |
| **Dead Letter Queue** | RabbitMQ DLX, Kafka with retry topics | Handle failed tasks gracefully. |
| **Idempotency Keys** | Custom header + Redis | Prevent duplicate message sending, duplicate payment processing. |
| **Webhook Delivery** | Custom `WebhookService` with retries + exponential backoff | Notify external systems (e.g., data repositories) about new publications. |
| **Scheduled Jobs** | Spring `@Scheduled`, Quartz | Daily digest emails, cleanup expired sessions, renew pre‑keys. |
| **Batch Processing** | Spring Batch | Process large CSV imports/exports in chunks. |

---

## 💾 5. Data & Caching

| Component | Technology | When to Use |
|-----------|------------|-------------|
| **Read‑Through / Write‑Through Cache** | Spring Cache abstraction + Redis | Cache frequently read data (user profiles, article metadata). |
| **Query Cache** | Hibernate Second‑Level Cache, Caffeine | Reduce SQL load for repeated queries. |
| **Database Indexing** | SQL `CREATE INDEX` | Speed up `WHERE` clauses, joins, sorting. Essential for large tables. |
| **Partitioning / Sharding** | PostgreSQL declarative partitioning, custom sharding | Split massive tables (e.g., messages) by date or user hash. |
| **Read Replicas** | Spring `@Transactional(readOnly=true)` + routing datasource | Offload analytic queries from master. |
| **Full‑Text Search** | Elasticsearch (sync via Change Data Capture) | Real‑time search across articles, messages, users. |
| **Time‑Series Data** | TimescaleDB (PostgreSQL extension) | Store and query article view events efficiently. |
| **Data Archival** | Custom job + cold storage (S3 Glacier) | Move old logs, deleted messages to cheap storage. |

---

## 🧪 6. Testing & CI/CD

| Phase | Tools | Why |
|-------|-------|-----|
| **Unit Testing** | JUnit 5, Mockito | Test business logic in isolation. |
| **Integration Testing** | Spring Boot Test, Testcontainers | Test with real database, RabbitMQ, Elasticsearch. |
| **API Contract Testing** | Pact | Ensure frontend and backend expectations match. |
| **End‑to‑End Testing** | Playwright, Cypress | Test critical user journeys (login, send message, publish article). |
| **Load Testing** | JMeter, Gatling, k6 | Simulate thousands of users; find bottlenecks. |
| **Security Testing** | OWASP ZAP, Burp Suite | Scan for common vulnerabilities (XSS, SQLi). |
| **CI/CD Pipeline** | GitHub Actions, GitLab CI, Jenkins | Automatically build, test, and deploy on every commit. |
| **Container Registry** | Docker Hub, GitHub Container Registry, AWS ECR | Store Docker images. |
| **Orchestration** | Kubernetes (k8s) | Manage deployments, scaling, rolling updates. |
| **Infrastructure as Code** | Terraform, Pulumi, AWS CloudFormation | Define servers, databases, load balancers as code. |
| **Blue‑Green / Canary Deployments** | Kubernetes with Argo Rollouts, AWS CodeDeploy | Zero‑downtime releases, rollback capability. |

---

## 🧾 7. Business & Compliance

| Component | Implementation | Purpose |
|-----------|----------------|---------|
| **GDPR / CCPA Compliance** | Data deletion endpoint, consent management | Let users request deletion of their data. |
| **Data Retention Policy** | Scheduled job | Automatically delete old logs, deleted messages after 30/90 days. |
| **Usage Billing** | Stripe, Lago (open‑source) | If you ever charge for premium features. |
| **Payment Webhooks** | Custom endpoint | Handle subscription events, invoices. |
| **Privacy Policy / Terms of Service** | Static pages + acceptance tracking | Legal requirement. |
| **Cookie Consent** | Frontend library + backend consent flag | Comply with EU cookie law. |
| **Data Export** | GDPR export endpoint (JSON/CSV) | Users can download all their data. |
| **Audit Logging** | Separate table + API | Required for ISO 27001, SOC2 certification. |

---

## 🧩 8. Feature Flags & Experimentation

| Component | Technology | Benefit |
|-----------|------------|---------|
| **Feature Flags** | LaunchDarkly, Flagsmith, custom (`FeatureFlagService`) | Enable/disable features without deployment. |
| **A/B Testing** | Custom assignment logic + metrics | Test new UI, recommendation algorithm variants. |
| **Gradual Rollout** | Flags + user percentage | Roll out a feature to 1% of users, then 10%, etc. |

---

## 🧠 9. Additional AI / ML Serving

| Component | Implementation | Use |
|-----------|----------------|-----|
| **Model Serving** | TensorFlow Serving, ONNX Runtime, BentoML | Serve recommendation model (GNN), classifier for smart notifications. |
| **Feature Store** | Feast (open‑source) | Manage features for ML models. |
| **Model Monitoring** | Evidently AI, WhyLogs | Detect data drift, model degradation. |
| **Async Inference** | Kafka + ML microservice | Process RAG queries, batch gap analysis offline. |

---

## 🌍 10. Localisation & Internationalisation (i18n)

| Component | Implementation |
|-----------|----------------|
| **Locale Detection** | `Accept-Language` header, user preference in DB |
| **Message Bundles** | Spring `MessageSource` + `.properties` files |
| **Date/Time Formatting** | User’s timezone stored in profile, use `ZonedDateTime` |
| **Right‑to‑Left (RTL) Support** | Frontend CSS; backend doesn’t need much. |

---

## 📦 11. Production Environment Checklist

- [ ] Domain name with SSL certificate
- [ ] CDN for static assets (React build, uploaded images)
- [ ] WAF (Web Application Firewall) – Cloudflare, AWS WAF
- [ ] Database backups (automated, encrypted, stored off‑site)
- [ ] Disaster Recovery plan (RTO / RPO)
- [ ] Log rotation and retention policy
- [ ] Security headers (HSTS, CSP, X‑Frame‑Options)
- [ ] Rate limiting for critical endpoints (login, register, send message)
- [ ] IP whitelist for admin endpoints
- [ ] Regular security audits (dependency updates, penetration testing)
- [ ] Compliance documentation (if handling medical/student data)

---

## ✅ Implementation Roadmap (What to Build First)

1. **Caching** – Redis for session store, rate limiting, and frequent queries.
2. **Async Tasks** – RabbitMQ for email sending, document indexing, PDF export.
3. **Full‑Text Search** – Elasticsearch for articles, messages, and users.
4. **Observability** – Prometheus + Grafana for metrics, Loki for logs.
5. **Security Hardening** – CSRF, rate limiting, JWT refresh tokens, HTTPS.
6. **CI/CD** – GitHub Actions to build, test, and deploy to staging.
7. **Production Environment** – Docker + Kubernetes on cloud (or self‑hosted).

---

## 🎯 Final Advice

You don’t need to implement everything at once. Start with **caching**, **async tasks**, **structured logging**, and **basic monitoring**. Then add Elasticsearch, a message queue, and Kubernetes as you scale. For a research platform, compliance (GDPR) and data durability are top priorities.