================================================================================ PYCURL - USER GUIDE (Android Python STB) - Generated by RIMI ================================================================================ Covers: what pycurl is, install/verify, Curl object, HTTP GET/POST, SSL, timeouts, write callbacks, and libcurl-impersonate features. Written for: Python 3.12 (RIMI build) on Android Version: pycurl 7.45.6 (libcurl-impersonate) Installed: /data/user/0/com.pythonstb.rimi/files/python/lib/python3.12/site-packages/ ================================================================================ -------------------------------------------------------------------------------- 1) WHAT IS PYCURL? -------------------------------------------------------------------------------- pycurl is a Python interface to libcurl, the multiprotocol file transfer library. This build uses libcurl-impersonate (a patched libcurl that mimics real browsers). Features: - HTTP, HTTPS, FTP, SFTP, SCP, and many other protocols - SSL/TLS with BoringSSL backend (via libcurl-impersonate) - Compression: zstd, brotli, gzip - HTTP/2 via nghttp2 - Browser impersonation (Chrome, Firefox, Safari headers) import pycurl print(pycurl.version) # libcurl/7.88.1 OpenSSL/3.1.4 ... brotli/1.0.9 ... -------------------------------------------------------------------------------- 2) INSTALL / VERIFY -------------------------------------------------------------------------------- Install (already done, but if you ever reinstall): pip install pycurl Quick smoke test: import pycurl c = pycurl.Curl() c.setopt(c.URL, "http://httpbin.org/get") buf = [] c.setopt(c.WRITEFUNCTION, buf.append) c.perform() print(c.getinfo(c.RESPONSE_CODE)) # 200 c.close() Expected output: 200 -------------------------------------------------------------------------------- 3) CURL OBJECT -------------------------------------------------------------------------------- import pycurl # Create a Curl handle (reusable for multiple requests) c = pycurl.Curl() # Set options c.setopt(pycurl.URL, "http://httpbin.org/get") c.setopt(pycurl.CONNECTTIMEOUT, 5) c.setopt(pycurl.TIMEOUT, 10) # Perform the request buf = [] c.setopt(pycurl.WRITEFUNCTION, buf.append) c.perform() # Get info print("status:", c.getinfo(pycurl.RESPONSE_CODE)) print("time:", c.getinfo(pycurl.TOTAL_TIME)) print("size:", c.getinfo(pycurl.SIZE_DOWNLOAD)) # Close when done c.close() Note: Curl handles can be reused. Create once, setopt different URLs, perform. -------------------------------------------------------------------------------- 4) HTTP GET -------------------------------------------------------------------------------- import pycurl from io import BytesIO c = pycurl.Curl() c.setopt(c.URL, "http://httpbin.org/get") c.setopt(c.CONNECTTIMEOUT, 5) c.setopt(c.TIMEOUT, 10) buf = BytesIO() c.setopt(c.WRITEDATA, buf) c.perform() print(buf.getvalue()) # response body print(c.getinfo(c.RESPONSE_CODE)) # 200 c.close() -------------------------------------------------------------------------------- 5) HTTP POST -------------------------------------------------------------------------------- import pycurl c = pycurl.Curl() c.setopt(c.URL, "http://httpbin.org/post") c.setopt(c.CONNECTTIMEOUT, 5) c.setopt(c.TIMEOUT, 10) # POST with form data c.setopt(c.POSTFIELDS, "name=Alice&age=30") c.setopt(c.HTTPHEADER, ["Content-Type: application/x-www-form-urlencoded"]) buf = [] c.setopt(c.WRITEFUNCTION, buf.append) c.perform() print("status:", c.getinfo(c.RESPONSE_CODE)) print("body:", "".join(buf)[:200]) c.close() # POST JSON import json data = json.dumps({"name": "Alice", "age": 30}) c2 = pycurl.Curl() c2.setopt(c2.URL, "http://httpbin.org/post") c2.setopt(c2.CONNECTTIMEOUT, 5) c2.setopt(c2.TIMEOUT, 10) c2.setopt(c2.POST, True) c2.setopt(c2.POSTFIELDS, data) c2.setopt(c2.HTTPHEADER, ["Content-Type: application/json"]) buf2 = [] c2.setopt(c2.WRITEFUNCTION, buf2.append) c2.perform() print("status:", c2.getinfo(c2.RESPONSE_CODE)) c2.close() -------------------------------------------------------------------------------- 6) HEADERS -------------------------------------------------------------------------------- import pycurl c = pycurl.Curl() c.setopt(c.URL, "http://httpbin.org/get") c.setopt(c.CONNECTTIMEOUT, 5) c.setopt(c.TIMEOUT, 10) # Custom headers c.setopt(c.HTTPHEADER, [ "User-Agent: MyApp/1.0", "Accept: application/json", "X-Custom: test" ]) # Capture response headers resp_headers = [] c.setopt(c.HEADERFUNCTION, lambda h: resp_headers.append(h.strip())) buf = [] c.setopt(c.WRITEFUNCTION, buf.append) c.perform() print("headers:", resp_headers[:5]) c.close() -------------------------------------------------------------------------------- 7) SSL / HTTPS -------------------------------------------------------------------------------- import pycurl c = pycurl.Curl() c.setopt(c.URL, "https://httpbin.org/get") c.setopt(c.CONNECTTIMEOUT, 5) c.setopt(c.TIMEOUT, 10) # SSL verification (enabled by default) c.setopt(c.SSL_VERIFYPEER, 1) c.setopt(c.SSL_VERIFYHOST, 2) buf = [] c.setopt(c.WRITEFUNCTION, buf.append) c.perform() print("status:", c.getinfo(c.RESPONSE_CODE)) ssl_verify = c.getinfo(c.SSL_VERIFYRESULT) print("SSL verify:", ssl_verify) c.close() # Disable verification (NOT recommended for production) c2 = pycurl.Curl() c2.setopt(c2.URL, "https://httpbin.org/get") c2.setopt(c2.SSL_VERIFYPEER, 0) c2.setopt(c2.SSL_VERIFYHOST, 0) -------------------------------------------------------------------------------- 8) TIMEOUTS -------------------------------------------------------------------------------- import pycurl c = pycurl.Curl() # Connection timeout (DNS + TCP connect) c.setopt(c.CONNECTTIMEOUT, 5) # 5 seconds # Total request timeout c.setopt(c.TIMEOUT, 30) # 30 seconds # DNS cache timeout c.setopt(c.DNS_CACHE_TIMEOUT, 300) # 5 minutes # Low speed limit (abort if speed < 1 byte/sec for 10 seconds) c.setopt(c.LOW_SPEED_LIMIT, 1) c.setopt(c.LOW_SPEED_TIME, 10) -------------------------------------------------------------------------------- 9) WRITE CALLBACKS -------------------------------------------------------------------------------- import pycurl from io import BytesIO # Method 1: WRITEFUNCTION callback def on_data(data): print("got %d bytes" % len(data)) c = pycurl.Curl() c.setopt(c.URL, "http://httpbin.org/get") c.setopt(c.WRITEFUNCTION, on_data) c.perform() c.close() # Method 2: WRITEDATA file-like object buf = BytesIO() c = pycurl.Curl() c.setopt(c.URL, "http://httpbin.org/get") c.setopt(c.WRITEDATA, buf) c.perform() print(buf.getvalue()[:100]) c.close() -------------------------------------------------------------------------------- 10) LIBCURL-IMPERSONATE FEATURES -------------------------------------------------------------------------------- This build uses libcurl-impersonate which supports browser impersonation: import pycurl c = pycurl.Curl() c.setopt(c.URL, "https://example.com") # Impersonate Chrome 99 c.setopt(c.HTTPHEADER, [ "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) " "AppleWebKit/537.36 (KHTML, like Gecko) " "Chrome/99.0.4844.51 Safari/537.36" ]) # The underlying libcurl-impersonate can also set TLS fingerprints # to match Chrome/Firefox/Safari behavior. Available impersonation targets (depends on libcurl-impersonate version): - Chrome 99, 100, 101, 104, 107, 110, 116, 119, 120 - Firefox 99, 102, 104, 113, 117, 118, 120 - Safari 15.3, 15.5, 15.6, 16.0 -------------------------------------------------------------------------------- 11) ERROR HANDLING -------------------------------------------------------------------------------- import pycurl # pycurl.error is raised on failures try: c = pycurl.Curl() c.setopt(c.URL, "http://localhost:1") c.setopt(c.CONNECTTIMEOUT, 2) c.perform() except pycurl.error as e: code, message = e.args print("error code:", code) # e.g. CURLE_COULDNT_CONNECT (7) print("message:", message) # e.g. "Failed to connect..." finally: c.close() Common error codes: CURLE_OK (0) - Success CURLE_COULDNT_RESOLVE (6) - DNS resolution failed CURLE_COULDNT_CONNECT (7) - TCP connect failed CURLE_OPERATION_TIMEDOUT (28) - Timeout CURLE_SSL_CONNECT (35) - SSL handshake failed -------------------------------------------------------------------------------- 12) PITFALLS & NOTES -------------------------------------------------------------------------------- - Always close Curl handles when done: c.close() - Reuse Curl handles for multiple requests (avoids DNS re-resolution) - pycurl.Curl() is NOT thread-safe. Use one Curl per thread. - WRITEFUNCTION receives bytes, not str - Response headers include the trailing \r\n - libcurl-impersonate uses BoringSSL, not OpenSSL - Built with static deps: BoringSSL, zstd, brotli, nghttp2 - Wheels are tagged cp312-cp312-android_24 - Network tests are skippable (may fail on restricted networks) - HTTPS requests require valid CA certs in the system store - POSTFIELDS is a string or bytes; use POSTFIELDSIZE for binary data ================================================================================ Generated by RIMI ================================================================================