Spaces:
Sleeping
Add comprehensive API Rate Limits & Throttling documentation
Browse filesAdded comprehensive API rate limiting documentation covering all major integration points:
**LinkedIn API:**
- Connection/messaging limits (100-200/week, 150 messages/day)
- API request limits by scale (7,500-30,000 requests/day)
- Exponential backoff implementation (15-30 min delays)
**Email APIs (SendGrid, Mailgun):**
- Request rate limits (300-600 requests/minute)
- Daily email quotas by plan tier
- Automatic throttling behavior
**Phone/SMS (Twilio):**
- 1 CPS limit per account/phone number
- Conversation limits (30 messages/30 seconds)
- Error codes and handling
**Data Enrichment (ZoomInfo, Clearbit, Hunter.io, Apollo.io):**
- Per-second and per-minute limits
- Daily quotas by plan
- Batch enrichment constraints
- Credit-based pricing models
**CRM (Salesforce):**
- 100,000+ daily requests (Enterprise)
- Concurrent request limits (25 max)
- Bulk API batch limits (15,000/day)
- Rolling 24-hour windows
**Platform Implementation:**
- Rate limit detection and monitoring
- Circuit breaker patterns
- Queue management
- Live Diagnostics panel integration
- Configuration file references
**Additional Resources:**
- Links to official API documentation for each service
- Best practices and recommended actions
- Monitoring and alerting guidelines
This documentation addresses the rate limit warning observed in Live Diagnostics and provides operational guidance for managing API quotas across all integrated services.
|
@@ -49,3 +49,165 @@ Example construction is provided in app.py showing default data structures and r
|
|
| 49 |
---
|
| 50 |
(legacy platform and API docs retained below)
|
| 51 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
---
|
| 50 |
(legacy platform and API docs retained below)
|
| 51 |
|
| 52 |
+
---
|
| 53 |
+
|
| 54 |
+
## API Rate Limits & Throttling
|
| 55 |
+
|
| 56 |
+
### Overview
|
| 57 |
+
|
| 58 |
+
This platform integrates with multiple external APIs for B2B outreach automation. Each service enforces rate limits to prevent abuse and ensure service stability. Understanding these limits is critical for:
|
| 59 |
+
- **Preventing service disruptions** (429 errors, temporary blocks)
|
| 60 |
+
- **Optimizing batch operations** (cadence scheduling, bulk enrichment)
|
| 61 |
+
- **Compliance monitoring** (audit trail accuracy)
|
| 62 |
+
- **Cost management** (avoiding overage charges)
|
| 63 |
+
|
| 64 |
+
**Rate Limit Reset Timing:** Most APIs reset limits at **midnight UTC** daily, with some using rolling windows.
|
| 65 |
+
|
| 66 |
+
---
|
| 67 |
+
|
| 68 |
+
### LinkedIn API Rate Limits
|
| 69 |
+
|
| 70 |
+
**Connection & Messaging Limits:**
|
| 71 |
+
- **Connection Requests:** 100-200 per week (varies by account reputation)
|
| 72 |
+
- New accounts: ~50-75/week
|
| 73 |
+
- Established accounts: ~100/week
|
| 74 |
+
- Trusted accounts (high SSI): up to 200/week
|
| 75 |
+
- **Messages:** 150 per day (premium accounts)
|
| 76 |
+
- **Profile Views:** 80/day (standard), up to 1,000/day (premium)
|
| 77 |
+
|
| 78 |
+
**API Request Limits (Scale-Dependent):**
|
| 79 |
+
- **1,000-5,000 members:** 7,500 requests/day, 1,500 requests/hour
|
| 80 |
+
- **5,001-10,000 members:** 15,000 requests/day, 15,000 requests/hour
|
| 81 |
+
- **10,001-20,000 members:** 30,000 requests/day, 30,000 requests/hour
|
| 82 |
+
- **Rate limit errors:** HTTP 429 "Too Many Requests"
|
| 83 |
+
- **Alert threshold:** Email notification at 75% of quota
|
| 84 |
+
|
| 85 |
+
**Best Practices:**
|
| 86 |
+
- Implement **exponential backoff** (15-30 minute delays) when hitting limits
|
| 87 |
+
- Monitor `X-RateLimit-Remaining` response headers
|
| 88 |
+
- Limits reset at midnight UTC daily
|
| 89 |
+
|
| 90 |
+
---
|
| 91 |
+
|
| 92 |
+
### Email API Rate Limits
|
| 93 |
+
|
| 94 |
+
#### SendGrid
|
| 95 |
+
- **API Rate Limit:** 600 requests/minute (varies by endpoint)
|
| 96 |
+
- **Free Plan:** 100 emails/day
|
| 97 |
+
- **Paid Plans:** Up to 1,500 emails/day, max 400 emails/hour
|
| 98 |
+
- **Enterprise:** 1.5M emails/month with dedicated IP
|
| 99 |
+
|
| 100 |
+
#### Mailgun
|
| 101 |
+
- **API Rate Limit:** 300 requests/minute
|
| 102 |
+
- **Free Plan:** 5,000 emails/month, 300 emails/day
|
| 103 |
+
- **Paid Plans:** 50,000-100,000 emails/month (Foundation/Growth/Scale)
|
| 104 |
+
- **Throttling:** Automatic rate limiting based on sender behavior
|
| 105 |
+
|
| 106 |
+
---
|
| 107 |
+
|
| 108 |
+
### Phone/SMS API Rate Limits (Twilio)
|
| 109 |
+
|
| 110 |
+
- **Outbound Calls:** 1 call per second (CPS) per account
|
| 111 |
+
- **SMS Messages:** 1 message per second per phone number
|
| 112 |
+
- **Conversation Limit:** 30 outbound replies between two numbers in 30 seconds
|
| 113 |
+
- **Error Code:** 14107 when SMS rate limit exceeded
|
| 114 |
+
- **Webhook Limits:** Service-specific (see Twilio Verify documentation)
|
| 115 |
+
|
| 116 |
+
**Important:** The 500th user in a batch will receive their call/SMS after ~500 seconds due to 1 CPS limit.
|
| 117 |
+
|
| 118 |
+
---
|
| 119 |
+
|
| 120 |
+
### Data Enrichment API Rate Limits
|
| 121 |
+
|
| 122 |
+
#### ZoomInfo
|
| 123 |
+
- **Standard APIs:** 25 requests/second (1,500 requests/minute)
|
| 124 |
+
- **Scaling APIs:** 33 requests/second
|
| 125 |
+
- **Daily Limits:**
|
| 126 |
+
- Pro Plan: ~1,000 API calls/day
|
| 127 |
+
- Enterprise Plan: up to 10,000 API calls/day
|
| 128 |
+
- **Batch Enrichment:** 100,000 records max per batch
|
| 129 |
+
- **Pricing:** Credit-based system (~$15,000/year for 5,000 credits)
|
| 130 |
+
|
| 131 |
+
#### Clearbit (Breeze Intelligence)
|
| 132 |
+
- **Rate Limits:** Not publicly specified (HubSpot integration)
|
| 133 |
+
- **Batch Enrichment:** Limited to 100,000 records at once
|
| 134 |
+
- **Pricing:** Credit-based, starts at $45-50/month (100 credits)
|
| 135 |
+
- **Real-time Updates:** API-first approach for live data
|
| 136 |
+
|
| 137 |
+
#### Hunter.io
|
| 138 |
+
- **Domain Search & Email Finder:** 15 requests/second, 500 requests/minute
|
| 139 |
+
- **Email Verifier:** 10 requests/second, 300 requests/minute
|
| 140 |
+
- **Error Response:** HTTP 403 when limit exceeded
|
| 141 |
+
- **Plans:**
|
| 142 |
+
- Free: Limited credits
|
| 143 |
+
- Growth: 10,000 credits/month
|
| 144 |
+
- Scale: 25,000 credits/month
|
| 145 |
+
|
| 146 |
+
#### Apollo.io
|
| 147 |
+
- **Free Plan:** 50 API calls/minute, 600 calls/day
|
| 148 |
+
- **Paid Plans:** 200 API calls/minute, 2,000 calls/day
|
| 149 |
+
- **Bulk Enrichment:** 1/10 of standard rate limit (but 10 inputs per call)
|
| 150 |
+
- **Error:** HTTP 429 after exceeding limit
|
| 151 |
+
|
| 152 |
+
---
|
| 153 |
+
|
| 154 |
+
### CRM API Rate Limits (Salesforce)
|
| 155 |
+
|
| 156 |
+
- **Daily Request Limit:** 100,000 base requests/24 hours (Enterprise Edition)
|
| 157 |
+
- Plus 1,000 additional requests per user license
|
| 158 |
+
- Example: 50 users = 150,000 daily requests
|
| 159 |
+
- **Concurrent Requests:** Max 25 long-running requests (20+ seconds)
|
| 160 |
+
- Developer orgs: 5 concurrent requests
|
| 161 |
+
- **API Timeout:** Hard 10-minute maximum per call
|
| 162 |
+
- **Bulk API:** 15,000 batches/day, 10,000 records per batch
|
| 163 |
+
- **Report API:** 1,200 async requests/hour
|
| 164 |
+
- **Measurement Window:** Rolling 24-hour window (not calendar days)
|
| 165 |
+
|
| 166 |
+
---
|
| 167 |
+
|
| 168 |
+
### Platform Implementation
|
| 169 |
+
|
| 170 |
+
**Our Rate Limit Handling Strategy:**
|
| 171 |
+
|
| 172 |
+
1. **Detection:** Monitor response headers (`X-RateLimit-*`, HTTP 429 status)
|
| 173 |
+
2. **Exponential Backoff:** 15-minute initial delay, doubling on repeated failures
|
| 174 |
+
3. **Circuit Breaker:** Temporary pause (15-30 min) when threshold reached
|
| 175 |
+
4. **Queue Management:** Batch operations respect per-service limits
|
| 176 |
+
5. **Alerting:** Live Diagnostics panel logs rate limit events
|
| 177 |
+
|
| 178 |
+
**Configuration Files:**
|
| 179 |
+
- Rate limit configs: `/docs/configs/rate_limits.json`
|
| 180 |
+
- Throttling logic: See `cadence_agent.py` backoff implementation
|
| 181 |
+
- Monitoring: Live Diagnostics Panel tracks limit-related warnings
|
| 182 |
+
|
| 183 |
+
**Example from Live Diagnostics:**
|
| 184 |
+
```
|
| 185 |
+
[2025-11-06T06:33:00Z] warning: Rate limit reached for LinkedIn API;
|
| 186 |
+
implementing exponential backoff for 15 minutes
|
| 187 |
+
```
|
| 188 |
+
|
| 189 |
+
---
|
| 190 |
+
|
| 191 |
+
### Monitoring & Alerts
|
| 192 |
+
|
| 193 |
+
**Dashboard Integration:**
|
| 194 |
+
- **Live Diagnostics Panel:** Real-time rate limit warnings
|
| 195 |
+
- **Audit Trail Export:** Historical rate limit incidents (CSV export)
|
| 196 |
+
- **Compliance Tracking:** Flags batches affected by rate limiting
|
| 197 |
+
|
| 198 |
+
**Recommended Actions:**
|
| 199 |
+
- Review Live Diagnostics panel regularly for rate limit patterns
|
| 200 |
+
- Adjust batch sizes and cadence timing based on observed limits
|
| 201 |
+
- Consider upgrading API plans if consistently hitting free tier limits
|
| 202 |
+
- Implement staggered scheduling to distribute load throughout the day
|
| 203 |
+
|
| 204 |
+
---
|
| 205 |
+
|
| 206 |
+
### Additional Resources
|
| 207 |
+
|
| 208 |
+
- LinkedIn API Documentation: [Microsoft Learn - Rate Limits](https://learn.microsoft.com/en-us/linkedin/shared/api-guide/concepts/rate-limits)
|
| 209 |
+
- Twilio Rate Limits: [Understanding Twilio Rate Limits](https://help.twilio.com/articles/115002943027)
|
| 210 |
+
- Salesforce API Limits: [API Request Limits and Allocations](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta)
|
| 211 |
+
- Hunter.io API: [Rate Limits Documentation](https://help.hunter.io/en/articles/1970956-hunter-api)
|
| 212 |
+
- Apollo.io API: [View API Usage Stats](https://docs.apollo.io/reference/view-api-usage-stats)
|
| 213 |
+
|