Spaces:
Running
Running
Deploy Aureon OS (deploy/selfhost stack: nginx edge + fusion API x2 + console + HNC organ + redis + postgres; paper mode) (part 8)
9607690 verified |
Download docs/integrations/COINAPI_INTEGRATION.md from GaryLeckey/aureon-os: direct link, hf CLI and curl.
- Browser
- Download file 17.7 kB
-
https://huggingface.co/spaces/GaryLeckey/aureon-os/resolve/main/docs/integrations/COINAPI_INTEGRATION.md
- Command line
-
hf download hf://spaces/GaryLeckey/aureon-os/docs/integrations/COINAPI_INTEGRATION.md
-
curl -L -o COINAPI_INTEGRATION.md https://huggingface.co/spaces/GaryLeckey/aureon-os/resolve/main/docs/integrations/COINAPI_INTEGRATION.md
17.7 kB
| # ๐โก CoinAPI Integration - Cross-Exchange Anomaly Detection | |
| **Status**: โ **INTEGRATED** | November 30, 2025 | |
| **Author**: Gary Leckey & GitHub Copilot | |
| --- | |
| ## ๐ฏ Overview | |
| The CoinAPI integration adds **cross-exchange anomaly detection** to our trading system, enabling us to: | |
| 1. **Detect Market Manipulation** - Price discrepancies across exchanges | |
| 2. **Identify Wash Trading** - Circular trades inflating volume | |
| 3. **Spot Orderbook Spoofing** - Fake liquidity manipulation | |
| 4. **Find Arbitrage Opportunities** - Real price differences to exploit | |
| 5. **Refine Algorithms Automatically** - Adapt thresholds based on anomalies | |
| ### The Core Insight | |
| > **"The Truth is in the Anomalies"** | |
| When exchange data disagrees, the discrepancy reveals the *real story*: | |
| - **Price manipulation** โ Avoid the asset temporarily | |
| - **Wash trading** โ Blacklist completely | |
| - **Orderbook spoofing** โ Require higher confidence | |
| - **Cross-exchange spreads** โ Arbitrage opportunity! | |
| --- | |
| ## ๐๏ธ Architecture | |
| ``` | |
| โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ | |
| โ COINAPI.IO (300+ Exchanges) โ | |
| โ Professional aggregated data: OHLCV, quotes, orderbooks โ | |
| โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ | |
| โ | |
| โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ | |
| โ CoinAPIClient โ | |
| โ - Rate-limited REST API calls โ | |
| โ - Caching for efficiency โ | |
| โ - Multi-exchange quote fetching โ | |
| โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ | |
| โ | |
| โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ | |
| โ AnomalyDetector โ | |
| โ Detects 9 types of anomalies: โ | |
| โ โข Price Manipulation (cross-exchange spread > 2%) โ | |
| โ โข Wash Trading (repeated identical prices) โ | |
| โ โข Orderbook Spoofing (70/30 bid/ask imbalance) โ | |
| โ โข Volume Inflation (3x normal volume) โ | |
| โ โข Latency Arbitrage (>500ms delays) โ | |
| โ โข Frontrunning (100ms windows) โ | |
| โ โข Liquidity Drain (sudden orderbook collapse) โ | |
| โ โข Exchange Outage (missing data) โ | |
| โ โข Cross-Exchange Spread (arbitrage opportunities) โ | |
| โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ | |
| โ | |
| โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ | |
| โ Algorithm Refinement Engine โ | |
| โ Applies automatic adjustments: โ | |
| โ โข Blacklists symbols (wash trading) โ | |
| โ โข Adjusts coherence thresholds (spoofing) โ | |
| โ โข Uses multi-exchange mean prices (arbitrage) โ | |
| โ โข Logs all refinements for analysis โ | |
| โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ | |
| โ | |
| โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ | |
| โ AurisEngine Integration โ | |
| โ โข scan_for_anomalies() - Periodic scanning โ | |
| โ โข is_symbol_blacklisted() - Filter opportunities โ | |
| โ โข get_coherence_adjustment() - Dynamic thresholds โ | |
| โ โข _apply_anomaly_refinement() - Auto-adjust rules โ | |
| โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ | |
| โ | |
| โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ | |
| โ find_opportunities() Pipeline โ | |
| โ 1. Scan for anomalies every 5 minutes โ | |
| โ 2. Filter blacklisted symbols โ | |
| โ 3. Apply coherence adjustments โ | |
| โ 4. Continue with HNC + Probability analysis โ | |
| โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ | |
| ``` | |
| --- | |
| ## ๐ง Configuration | |
| ### Environment Variables | |
| ```bash | |
| # Required for live anomaly detection | |
| export COINAPI_KEY=your-api-key-here | |
| export ENABLE_COINAPI=1 | |
| ``` | |
| ### CONFIG Parameters | |
| ```python | |
| CONFIG = { | |
| # CoinAPI Anomaly Detection | |
| 'ENABLE_COINAPI': False, # Requires API key | |
| 'COINAPI_SCAN_INTERVAL': 300, # Scan every 5 minutes | |
| 'COINAPI_MIN_SEVERITY': 0.40, # Act on anomalies โฅ40% severity | |
| 'COINAPI_BLACKLIST_DURATION': 3600, # Blacklist for 1 hour | |
| 'COINAPI_ADJUST_COHERENCE': True, # Auto-adjust thresholds | |
| 'COINAPI_PRICE_SOURCE': 'multi_exchange', # Use aggregated prices | |
| } | |
| ``` | |
| --- | |
| ## ๐ Anomaly Types & Refinements | |
| | Anomaly Type | Detection Method | Automatic Refinement | | |
| |--------------|------------------|----------------------| | |
| | ๐ฐ **Price Manipulation** | Price >2% from cross-exchange mean | Increase coherence threshold +0.1<br>Reduce position size ร0.5 | | |
| | ๐ **Wash Trading** | >15% trades at identical prices | **Blacklist for 1 hour**<br>Position size ร0.0 | | |
| | ๐ **Orderbook Spoofing** | Bid/ask ratio >70/30 | Delay entry +60s<br>Reduce position size ร0.7 | | |
| | ๐ **Cross-Exchange Spread** | >2% arbitrage opportunity | Use multi-exchange mean price<br>**Increase position size ร1.2** | | |
| | โก **Latency Arbitrage** | >500ms delays | Compensate for latency<br>Reduce position size ร0.8 | | |
| | ๐ **Volume Inflation** | Volume >3ร normal | Require higher volume confirmation | | |
| | ๐ฏ **Frontrunning** | Suspicious <100ms patterns | Delay execution slightly | | |
| | ๐ง **Liquidity Drain** | Orderbook collapse | Avoid until liquidity returns | | |
| | ๐จ **Exchange Outage** | Missing data feeds | Skip exchange temporarily | | |
| --- | |
| ## ๐ Usage | |
| ### Basic Integration (Disabled by Default) | |
| ```python | |
| from aureon_unified_ecosystem import AureonKrakenEcosystem | |
| # Without CoinAPI (default) | |
| eco = AureonKrakenEcosystem(initial_balance=200.0, dry_run=True) | |
| # Anomaly detection is disabled but methods are available | |
| ``` | |
| ### With CoinAPI Key | |
| ```python | |
| import os | |
| os.environ['COINAPI_KEY'] = 'your-key-here' | |
| os.environ['ENABLE_COINAPI'] = '1' | |
| from aureon_unified_ecosystem import AureonKrakenEcosystem | |
| eco = AureonKrakenEcosystem(initial_balance=200.0, dry_run=True) | |
| # CoinAPI anomaly detection is now ACTIVE | |
| ``` | |
| ### Manual Anomaly Scanning | |
| ```python | |
| # Scan specific symbols for anomalies | |
| symbols = ['BTC/USD', 'ETH/USD', 'BNB/USD'] | |
| anomalies = eco.auris.scan_for_anomalies(symbols) | |
| for anom in anomalies: | |
| print(f"Detected: {anom['type']}") | |
| print(f"Severity: {anom['severity']:.0%}") | |
| print(f"Recommendation: {anom['recommendation']}") | |
| ``` | |
| ### Check Blacklist Status | |
| ```python | |
| # Check if symbol is blacklisted | |
| if eco.auris.is_symbol_blacklisted('BTC/USD'): | |
| print("BTC/USD is blacklisted due to anomaly") | |
| # Get coherence adjustment | |
| adjustment = eco.auris.get_coherence_adjustment('ETH/USD') | |
| print(f"ETH/USD coherence threshold: ร{adjustment:.2f}") | |
| ``` | |
| --- | |
| ## ๐ Integration with HNC + Probability Matrix | |
| The CoinAPI layer works seamlessly with existing systems: | |
| ``` | |
| 1. CoinAPI Anomaly Detection | |
| โ (Filters & Adjusts) | |
| 2. Opportunity Filtering | |
| โ (Blacklist check) | |
| 3. HNC Frequency Analysis | |
| โ (Harmonic bonus/penalty) | |
| 4. Probability Matrix | |
| โ (2-hour forecast) | |
| 5. Final Score & Position Sizing | |
| โ | |
| 6. Trade Execution | |
| ``` | |
| ### Example Flow | |
| ```python | |
| # Symbol goes through complete pipeline: | |
| # 1. CoinAPI detects orderbook spoofing on XYZ/USD | |
| # โ Coherence threshold increased to 0.75 (from 0.65) | |
| # 2. Opportunity filtering | |
| # โ XYZ/USD requires 0.75 coherence instead of 0.65 | |
| # 3. HNC analysis shows 528Hz harmonic | |
| # โ +15 score bonus, ร1.15 position size | |
| # 4. Probability matrix shows 88% probability | |
| # โ +20 score bonus | |
| # 5. Final score: 145 | |
| # โ Passes thresholds despite higher coherence requirement | |
| # โ Position opened with adjusted size | |
| ``` | |
| --- | |
| ## ๐งช Testing | |
| ### Run Comprehensive Test Suite | |
| ```bash | |
| python test_coinapi_integration.py | |
| ``` | |
| Tests include: | |
| - โ Blacklist functionality | |
| - โ Coherence adjustments | |
| - โ Anomaly refinement logic | |
| - โ Opportunity filtering | |
| - โ Full system integration | |
| ### Standalone CoinAPI Demo | |
| ```bash | |
| python coinapi_anomaly_detector.py | |
| ``` | |
| Demonstrates: | |
| - Cross-exchange price analysis | |
| - Anomaly detection with simulated data | |
| - Algorithm refinement recommendations | |
| --- | |
| ## ๐ฐ CoinAPI Free Tier | |
| | Feature | Limit | | |
| |---------|-------| | |
| | **Requests/Day** | 100 | | |
| | **Exchanges** | 300+ | | |
| | **Data Types** | OHLCV, Quotes, Orderbooks, Trades | | |
| | **Historical** | Limited | | |
| | **WebSocket** | No (REST only) | | |
| ### Optimization Strategy | |
| With 100 requests/day and 5-minute scans: | |
| - **288 potential scans/day** (1 request per scan) | |
| - **Scan 3-5 symbols per scan** (15-25 symbols covered) | |
| - **Rotate through top opportunities** (not all symbols) | |
| - **Cache results aggressively** (minimize duplicate calls) | |
| --- | |
| ## ๐ Real-World Example | |
| ### Detected Anomaly Report | |
| ``` | |
| โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ | |
| โ ๐โก COINAPI ANOMALY REPORT: BTC/USD โ | |
| โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฃ | |
| โ Exchanges Analyzed: 5 โ | |
| โ Mean Price: $69420.50 โ | |
| โ Price StdDev: $350.25 โ | |
| โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฃ | |
| โ DETECTED ANOMALIES: โ | |
| โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฃ | |
| โ 1. ๐ฐ Price Manipulation โ | |
| โ Severity: โโโโโโโโโโ 75% โ | |
| โ Price 4.2% away from cross-exchange mean โ | |
| โ โ AVOID โ | |
| โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฃ | |
| โ 2. ๐ Cross-Exchange Spread โ | |
| โ Severity: โโโโโโโโโโ 45% โ | |
| โ 2.8% arbitrage spread between exchanges โ | |
| โ โ ARBITRAGE OPPORTUNITY โ | |
| โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฃ | |
| โ ALGORITHM REFINEMENTS: โ | |
| โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฃ | |
| โ 1. Price manipulation detected - require higher confidence โ | |
| โ โข coherence_threshold: +0.1 โ | |
| โ โข position_size: ร0.5 โ | |
| โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฃ | |
| โ 2. Arbitrage opportunity - use aggregated price โ | |
| โ โข price_source: multi_exchange_mean โ | |
| โ โข position_size: ร1.2 โ | |
| โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ | |
| ``` | |
| --- | |
| ## ๐ฏ Key Benefits | |
| ### 1. **Data Quality Validation** | |
| - Cross-validates exchange data | |
| - Detects feed manipulation | |
| - Uses professional-grade aggregated data | |
| ### 2. **Adaptive Algorithm** | |
| - Automatically adjusts to market conditions | |
| - Learns from anomalies | |
| - Refines thresholds in real-time | |
| ### 3. **Risk Reduction** | |
| - Avoids manipulated markets | |
| - Identifies wash trading | |
| - Detects spoofed liquidity | |
| ### 4. **Opportunity Enhancement** | |
| - Finds arbitrage opportunities | |
| - Uses better pricing data | |
| - Exploits market inefficiencies | |
| --- | |
| ## ๐ฎ Future Enhancements | |
| 1. **Historical Anomaly Analysis** | |
| - Track anomaly patterns over time | |
| - Machine learning on refinement effectiveness | |
| 2. **Exchange Reputation Scoring** | |
| - Rate exchanges by data quality | |
| - Prefer reliable exchanges | |
| 3. **Real-Time WebSocket Integration** | |
| - Upgrade to paid tier for WebSocket | |
| - Sub-second anomaly detection | |
| 4. **Cross-Asset Correlation** | |
| - Detect manipulation across asset pairs | |
| - Systemic risk indicators | |
| --- | |
| ## ๐ Files | |
| | File | Purpose | | |
| |------|---------| | |
| | `coinapi_anomaly_detector.py` | Core anomaly detection engine | | |
| | `test_coinapi_integration.py` | Comprehensive test suite | | |
| | `aureon_unified_ecosystem.py` | Integration into main system | | |
| | `COINAPI_INTEGRATION.md` | This documentation | | |
| --- | |
| ## ๐ Success Metrics | |
| Track these metrics to measure CoinAPI value: | |
| - **Anomalies Detected** - Total count by type | |
| - **Symbols Blacklisted** - Wash trading avoidance | |
| - **Refinements Applied** - Algorithm adjustments | |
| - **Arbitrage Opportunities** - Cross-exchange spreads found | |
| - **False Positive Rate** - Good trades filtered incorrectly | |
| - **Performance Impact** - Win rate before/after CoinAPI | |
| --- | |
| ## ๐ Conclusion | |
| CoinAPI integration adds a **professional data validation layer** to our trading system. By detecting anomalies in cross-exchange data, we: | |
| โ Avoid manipulated markets | |
| โ Identify real arbitrage opportunities | |
| โ Automatically refine our algorithms | |
| โ Use higher-quality aggregated pricing | |
| โ Reduce false signals and bad trades | |
| The system now combines: | |
| 1. **HNC Frequency Analysis** (Solfeggio harmonics) | |
| 2. **Probability Matrix** (2-hour temporal forecasting) | |
| 3. **CoinAPI Anomaly Detection** (cross-exchange validation) | |
| **Result**: A self-improving trading algorithm that learns from market anomalies and adapts in real-time. | |
| --- | |
| *"The Truth is in the Anomalies"* | |
| **Gary Leckey | November 30, 2025** | |