File size: 2,544 Bytes
ed12f96 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | # Profiling
In slime, we can perform detailed performance analysis of the rollout process using the profiling interface provided by SGLang.
## 1. Sleeping the Rollout Process
For more flexible stress testing and profiling, it is often useful to make the slime rollout process enter a waiting state after initialization, instead of starting generation immediately.
You can achieve this by replacing the `rollout_function_path` in your startup arguments without modifying the source code:
```bash
python train.py \
--rollout-function-path slime.rollout.sleep_rollout.sleep \
... (other arguments)
```
This function will make the rollout process enter an infinite wait loop, allowing you to manually send requests or run stress testing tools.
## 2. Obtaining SGLang Engine List
SGLang engines (workers) are registered with the router. You can retrieve the list of all active engines by accessing the `/workers` endpoint of the router.
The router address is typically printed in the startup logs:
```
Router launched at 127.0.0.1:3000
```
You can use `curl` to view the workers:
```bash
curl http://127.0.0.1:3000/workers
```
## 3. Using Automated Profiling Tool
To simplify profiling across multiple engines simultaneously, we provide an automated script: `tools/profile_rollout.py`.
### Starting Profiling
By default, this tool starts profiling on all workers and will automatically stop after 3 steps:
```bash
python tools/profile_rollout.py --router-url http://127.0.0.1:3000 --action start --num-steps 3
```
**Key Parameters:**
* `--router-url`: The URL of the Router.
* `--num-steps`: Number of steps to record, defaults to 3.
* `--output-dir`: Directory where trace files will be saved.
* `--activities`: Activities to monitor, e.g., `GPU` `CPU`.
* `--profile-by-stage`: Whether to profile by stage (prefill/decode).
### Stopping Profiling Manually
If you did not set `num_steps` or wish to stop early:
```bash
python tools/profile_rollout.py --router-url http://127.0.0.1:3000 --action stop
```
## 4. Running Stress Tests
While the Rollout process is in a waiting state via `sleep_rollout`, you can:
1. Start profiling using `tools/profile_rollout.py`.
2. Use stress testing tools (such as SGLang's built-in benchmark tools) to send requests to the router or directly to the engines.
3. Wait for profiling to complete (if `num_steps` was set) or stop it manually.
4. Collect the `.json` trace files from the `output_dir` and view them using `chrome://tracing` in Chrome or [Perfetto](https://ui.perfetto.dev/).
|