# Performance Optimization Report ## Context The credit scoring API was already deployed with structured prediction logging, drift monitoring, and operational latency tracking. The optimization objective was to reduce inference latency without changing model outputs or the API contract. Runtime before optimization: - API: FastAPI - Model: MLflow LightGBM model `models:/home_credit_scoring/1` - Monitoring: prediction logging, drift monitoring, Evidently 0.7.21 - Hardware for local tests: developer workstation CPU ## Baseline Measurement The initial local inference benchmark measured the model hot path with the existing pandas `DataFrame` input. ```text Runs: 100 Mean latency: 1.9436 ms Median latency: 1.8421 ms Min latency: 1.4010 ms Max latency: 3.6444 ms ``` The benchmark excluded network overhead and focused on local model inference. ## Profiling Results `cProfile` was used to profile 1000 repeated predictions after model loading. This step was run before testing ONNX to avoid optimizing blindly. At that stage, the production path used the LightGBM Python runtime with pandas input, so the goal was to identify whether latency came from the model computation, input preparation, API overhead, logging, or monitoring. Top cumulative-time entries: ```text 1000 calls main.py:55(predict_default_probability) 3.648 s 1000 calls sklearn.py:1615(predict_proba) 3.643 s 1000 calls sklearn.py:1093(predict) 3.599 s 1000 calls basic.py:4701(predict) 2.572 s 1000 calls basic.py:827(_data_from_pandas) 1.996 s 1000 calls basic.py:810(_pandas_to_numpy) 1.328 s 1000 calls basic.py:798(_check_for_bad_pandas_dtypes) 1.204 s ``` The main bottleneck was not the tree inference itself, but the pandas-to-LightGBM input conversion and dtype validation. The initial `cProfile` results did not show ONNX-related gains because ONNX Runtime was not yet part of the inference path. `cProfile` is also most useful for Python-level function calls; native runtimes such as ONNX Runtime execute most work inside optimized C/C++ code and usually appear as a compact `session.run(...)` call from Python. For that reason, profiling was used to understand the existing LightGBM path, while benchmarks were used to compare runtime alternatives. ## Optimization Tested The optimized version builds a NumPy array directly from the validated `CreditApplication`, using the exact `MODEL_FEATURES` order expected by the model. Before: ```text CreditApplication -> pandas DataFrame -> LightGBM predict_proba ``` After: ```text CreditApplication -> NumPy array -> LightGBM predict_proba ``` The API still validates inputs with Pydantic and still logs predictions and drift monitoring records. Only the internal model input representation changed. ## Accuracy And Regression Check The DataFrame and NumPy paths were compared on multiple valid API inputs. ```text Case 1 diff: 0.000000000000 Case 2 diff: 0.000000000000 Case 3 diff: 0.000000000000 Case 4 diff: 0.000000000000 Case 5 diff: 0.000000000000 All cases matched. Max probability diff: 0.000000000000 ``` Predicted classes and probabilities were identical for the tested cases. ## NumPy Benchmark The final comparison was run in the same Python process to reduce noise between runs. ```text DataFrame benchmark mean: 1.5190 ms median: 1.4541 ms min: 1.3538 ms max: 2.4473 ms NumPy benchmark mean: 1.0146 ms median: 0.9700 ms min: 0.8932 ms max: 2.1252 ms Mean latency improvement: 33.21% ``` The optimized hot-path benchmark with 1000 measured runs produced: ```text Runs: 1000 Mean latency: 0.9596 ms Median latency: 0.9230 ms Min latency: 0.8809 ms Max latency: 1.6724 ms ``` ## ONNX Runtime Evaluation The LightGBM model was exported to ONNX with `onnxmltools` and `target_opset=15`. Opset 15 was selected because the installed LightGBM converter supports ONNX opsets up to 15 for this model type. ONNX Runtime was then benchmarked against the optimized LightGBM NumPy path in the same Python process. ```text LightGBM NumPy benchmark mean: 0.9951 ms median: 0.9348 ms min: 0.8677 ms max: 2.1814 ms ONNX Runtime benchmark mean: 0.0305 ms median: 0.0271 ms min: 0.0238 ms max: 0.9632 ms ONNX mean latency improvement vs LightGBM NumPy: 96.94% ``` Prediction equivalence was checked on five valid API inputs. ```text Max probability diff: 0.000000131269 Tolerance: 1e-05 Predicted classes: identical ``` The small probability differences are expected numerical differences between the LightGBM Python runtime and ONNX Runtime. They remained far below the validation tolerance and did not change predicted classes. The ONNX benchmark measures the model hot path only. The full `/predict` HTTP latency also includes FastAPI routing, Pydantic validation, authentication, JSON serialization, logging, drift monitoring, and network overhead. Therefore, the full API response time is expected to improve less than the pure model runtime, even though the model inference step itself is much faster. ## Monitoring Notes Evidently was enabled with the modern 0.7 API: ```python from evidently import Report from evidently.presets import DataDriftPreset ``` A local drift check confirmed: ```text evidently_available: True batch_size: 30 ``` Small synthetic batches may produce NumPy statistical warnings during drift tests. Production drift monitoring should use sufficiently large batches. ## Validation Automated tests after ONNX integration: ```text 23 passed, 18 warnings ``` The warnings are emitted by Evidently/NumPy drift calculations on small test batches and are not related to the optimized inference path. ## Final Configuration The final configuration uses ONNX Runtime as the default inference runtime and keeps LightGBM available as a rollback/fallback option. Runtime selection is controlled by: ```text MODEL_RUNTIME=onnx MODEL_RUNTIME=lightgbm ``` The ONNX model path can be overridden with: ```text ONNX_MODEL_PATH ``` ONNX Runtime was selected because: - profiling first identified pandas conversion as the initial bottleneck; - replacing pandas with NumPy reduced LightGBM inference latency by about 33%; - ONNX Runtime then reduced the optimized model hot-path latency by about 97%; - prediction classes stayed identical across the validation cases; - maximum observed probability difference was only `0.000000131269`; - LightGBM remains available through `MODEL_RUNTIME=lightgbm` if rollback is needed. This configuration improves response-time headroom while preserving the existing API contract, monitoring pipeline, and CI/CD deployment flow.