Spaces:
Paused
Paused
File size: 6,062 Bytes
93dd899 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 | ---
title: Backend Testing
sidebar_position: 3
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Backend Testing
🚧 **Coming Soon** 🚧
Complete guide for testing Superset's Python backend, APIs, and database interactions.
## Topics to be covered:
- Pytest configuration and fixtures
- Unit testing best practices
- Integration testing with databases
- API endpoint testing
- Mocking strategies and patterns
- Testing async operations with Celery
- Security testing guidelines
- Performance and load testing
- Test database setup and teardown
- Coverage requirements
## Quick Commands
```bash
# Run all backend tests
pytest
# Run specific test file
pytest tests/unit_tests/specific_test.py
# Run with coverage
pytest --cov=superset
# Run tests in parallel
pytest -n auto
# Run only unit tests
pytest tests/unit_tests/
# Run only integration tests
pytest tests/integration_tests/
```
## Testing Alerts & Reports with Celery and MailHog
The Alerts & Reports feature relies on Celery for task scheduling and execution. To test it locally, you need Redis (message broker), Celery Beat (scheduler), a Celery Worker (executor), and an SMTP server to receive email notifications.
### Prerequisites
- Redis running on `localhost:6379`
- [MailHog](https://github.com/mailhog/MailHog) installed (a local SMTP server with a web UI for viewing caught emails)
### superset_config.py
Your `CeleryConfig` **must** include `beat_schedule`. When you define a custom `CeleryConfig` class in `superset_config.py`, it replaces the default entirely. If you omit `beat_schedule`, Celery Beat will start but never schedule any report tasks.
```python
from celery.schedules import crontab
from superset.tasks.types import ExecutorType
REDIS_HOST = "localhost"
REDIS_PORT = "6379"
class CeleryConfig:
broker_url = f"redis://{REDIS_HOST}:{REDIS_PORT}/0"
result_backend = f"redis://{REDIS_HOST}:{REDIS_PORT}/0"
broker_connection_retry_on_startup = True
imports = (
"superset.sql_lab",
"superset.tasks.scheduler",
"superset.tasks.thumbnails",
"superset.tasks.cache",
)
worker_prefetch_multiplier = 10
task_acks_late = True
beat_schedule = {
"reports.scheduler": {
"task": "reports.scheduler",
"schedule": crontab(minute="*", hour="*"),
},
"reports.prune_log": {
"task": "reports.prune_log",
"schedule": crontab(minute=0, hour=0),
},
}
CELERY_CONFIG = CeleryConfig
# SMTP settings pointing to MailHog
SMTP_HOST = "localhost"
SMTP_PORT = 1025
SMTP_STARTTLS = False
SMTP_SSL = False
SMTP_USER = ""
SMTP_PASSWORD = ""
SMTP_MAIL_FROM = "superset@localhost"
# Must match where your frontend is running
WEBDRIVER_BASEURL = "http://localhost:9000/"
ALERT_REPORTS_EXECUTE_AS = [ExecutorType.OWNER]
FEATURE_FLAGS = {
"ALERT_REPORTS": True,
# Recommended for better screenshot support (WebGL/DeckGL charts)
"PLAYWRIGHT_REPORTS_AND_THUMBNAILS": True,
}
```
:::note
Do not include `"superset.tasks.async_queries"` in `CeleryConfig.imports` unless you need Global Async Queries. That module accesses `current_app.config` at import time and will crash the worker with a "Working outside of application context" error.
:::
### Starting the Services
Start MailHog, then Celery Beat and Worker in separate terminals:
```bash
# Terminal 1 - MailHog (SMTP on :1025, Web UI on :8025)
MailHog
# Terminal 2 - Celery Beat (scheduler)
celery --app=superset.tasks.celery_app:app beat --loglevel=info
# Terminal 3 - Celery Worker (executor)
celery --app=superset.tasks.celery_app:app worker --concurrency=1 --loglevel=info
```
Use `--concurrency=1` to limit resource usage on your dev machine.
### Verifying the Setup
1. **Beat** should log `Scheduler: Sending due task reports.scheduler (reports.scheduler)` once per minute
2. **Worker** should log `Scheduling alert <name> eta: <timestamp>` for each active report
3. Create a test report in **Settings > Alerts & Reports** with a `* * * * *` cron schedule
4. Check **http://localhost:8025** (MailHog web UI) for the email within 1-2 minutes
### Troubleshooting
| Problem | Solution |
| ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| Beat shows no output | Ensure `beat_schedule` is defined in your `CeleryConfig` and `--loglevel=info` is set |
| "Report Schedule is still working, refusing to re-compute" | Previous executions are stuck. Reset with: `UPDATE report_schedule SET last_state = 'Not triggered' WHERE id = <id>;` |
| Task backlog overwhelming the worker | Flush Redis: `redis-cli FLUSHDB`, then restart Beat and Worker |
| Screenshot timeout | Ensure your frontend dev server is running and `WEBDRIVER_BASEURL` matches its URL |
---
_This documentation is under active development. Check back soon for updates!_
|