Spaces:
Sleeping
Sleeping
|
Download apps/ruby-sdk/README.md from s-eto-hub/firecrawl-api: direct link, hf CLI and curl.
- Browser
- Download file 4.87 kB
-
https://huggingface.co/spaces/s-eto-hub/firecrawl-api/resolve/main/apps/ruby-sdk/README.md
- Command line
-
hf download hf://spaces/s-eto-hub/firecrawl-api/apps/ruby-sdk/README.md
-
curl -L -o README.md https://huggingface.co/spaces/s-eto-hub/firecrawl-api/resolve/main/apps/ruby-sdk/README.md
4.87 kB
| # Firecrawl Ruby SDK | |
| Ruby SDK for the [Firecrawl](https://firecrawl.dev) v2 web scraping API. | |
| ## Prerequisites | |
| - Ruby >= 3.0 | |
| ## Installation | |
| Add to your `Gemfile`: | |
| ```ruby | |
| gem "firecrawl-sdk", "~> 1.5" | |
| ``` | |
| Or install directly: | |
| ```bash | |
| gem install firecrawl-sdk | |
| ``` | |
| ## Quick Start | |
| ```ruby | |
| require "firecrawl" | |
| # Create a client | |
| client = Firecrawl::Client.new(api_key: "fc-your-api-key") | |
| # Or load from FIRECRAWL_API_KEY environment variable | |
| client = Firecrawl::Client.from_env | |
| # Scrape a single page | |
| doc = client.scrape("https://example.com") | |
| puts doc.markdown | |
| ``` | |
| ## Environment Setup | |
| ```bash | |
| export FIRECRAWL_API_KEY="fc-your-api-key" | |
| # Optional: custom API URL | |
| export FIRECRAWL_API_URL="http://localhost:3002" | |
| ``` | |
| ## API Reference | |
| ### Scrape | |
| ```ruby | |
| # Basic scrape | |
| doc = client.scrape("https://example.com") | |
| puts doc.markdown | |
| # Scrape with options | |
| doc = client.scrape("https://example.com", | |
| Firecrawl::Models::ScrapeOptions.new( | |
| formats: ["markdown", "html"], | |
| only_main_content: true, | |
| wait_for: 1000 | |
| )) | |
| puts doc.html | |
| ``` | |
| ### Video Extraction | |
| Use the `video` format on supported video URLs, including YouTube and TikTok. The returned `video` field is a signed URL to the extracted video file. | |
| ```ruby | |
| doc = client.scrape("https://www.youtube.com/watch?v=dQw4w9WgXcQ", | |
| Firecrawl::Models::ScrapeOptions.new(formats: ["video"])) | |
| puts doc.video | |
| ``` | |
| ### Parse | |
| Upload a local file (`html`, `pdf`, `docx`, etc.) via multipart form data and | |
| parse it synchronously. Parse options intentionally exclude browser-only | |
| features such as change tracking, screenshot, branding, audio, video, actions, | |
| wait_for, location, and mobile. The `proxy` option only accepts `"auto"` or `"basic"`. | |
| ```ruby | |
| # From disk | |
| file = Firecrawl::Models::ParseFile.from_path("./document.pdf") | |
| # Or from memory | |
| file = Firecrawl::Models::ParseFile.new( | |
| filename: "upload.html", | |
| content: "<html>hi</html>", | |
| content_type: "text/html" | |
| ) | |
| doc = client.parse(file, | |
| Firecrawl::Models::ParseOptions.new(formats: ["markdown"])) | |
| puts doc.markdown | |
| ``` | |
| ### Crawl | |
| ```ruby | |
| # Crawl with auto-polling (blocks until complete) | |
| job = client.crawl("https://example.com", | |
| Firecrawl::Models::CrawlOptions.new(limit: 50)) | |
| job.data.each { |doc| puts doc.markdown } | |
| # Async crawl | |
| response = client.start_crawl("https://example.com", | |
| Firecrawl::Models::CrawlOptions.new(limit: 10)) | |
| puts response.id | |
| # Check status | |
| status = client.get_crawl_status(response.id) | |
| puts status.status | |
| # Cancel | |
| client.cancel_crawl(response.id) | |
| ``` | |
| ### Batch Scrape | |
| ```ruby | |
| urls = ["https://example.com/page1", "https://example.com/page2"] | |
| # Batch scrape with auto-polling | |
| job = client.batch_scrape(urls, | |
| Firecrawl::Models::BatchScrapeOptions.new( | |
| options: Firecrawl::Models::ScrapeOptions.new(formats: ["markdown"]) | |
| )) | |
| job.data.each { |doc| puts doc.markdown } | |
| ``` | |
| ### Map | |
| ```ruby | |
| # Discover URLs on a website | |
| result = client.map("https://example.com") | |
| result.links.each { |link| puts link["url"] } | |
| # With options | |
| result = client.map("https://example.com", | |
| Firecrawl::Models::MapOptions.new(limit: 100, search: "blog")) | |
| ``` | |
| ### Search | |
| ```ruby | |
| # Web search | |
| results = client.search("firecrawl web scraping") | |
| results.web&.each { |r| puts r["url"] } | |
| # With options | |
| results = client.search("latest news", | |
| Firecrawl::Models::SearchOptions.new(limit: 5, location: "US")) | |
| ``` | |
| ### Agent | |
| ```ruby | |
| # Run an AI agent task (blocks until complete) | |
| status = client.agent( | |
| Firecrawl::Models::AgentOptions.new( | |
| prompt: "Find the pricing information", | |
| urls: ["https://example.com"] | |
| )) | |
| puts status.data | |
| ``` | |
| ### Usage & Metrics | |
| ```ruby | |
| # Check concurrency | |
| concurrency = client.get_concurrency | |
| puts concurrency.concurrency | |
| # Check credit usage | |
| usage = client.get_credit_usage | |
| puts usage.remaining_credits | |
| ``` | |
| ## Configuration | |
| ```ruby | |
| client = Firecrawl::Client.new( | |
| api_key: "fc-your-api-key", | |
| api_url: "https://api.firecrawl.dev", # custom API URL | |
| timeout: 300, # HTTP timeout in seconds | |
| max_retries: 3, # automatic retries | |
| backoff_factor: 0.5 # exponential backoff factor | |
| ) | |
| ``` | |
| ## Error Handling | |
| ```ruby | |
| begin | |
| doc = client.scrape("https://example.com") | |
| rescue Firecrawl::AuthenticationError => e | |
| puts "Invalid API key: #{e.message}" | |
| rescue Firecrawl::RateLimitError => e | |
| puts "Rate limited: #{e.message}" | |
| rescue Firecrawl::JobTimeoutError => e | |
| puts "Job #{e.job_id} timed out after #{e.timeout_seconds}s" | |
| rescue Firecrawl::FirecrawlError => e | |
| puts "Error (#{e.status_code}): #{e.message}" | |
| end | |
| ``` | |
| ## Development | |
| ### Building from Source | |
| ```bash | |
| cd apps/ruby-sdk | |
| bundle install | |
| ``` | |
| ### Running Tests | |
| ```bash | |
| # Unit tests | |
| bundle exec rake test | |
| # With API key for E2E tests | |
| FIRECRAWL_API_KEY=fc-your-key bundle exec rake test | |
| ``` | |
| ## License | |
| MIT License - see [LICENSE](LICENSE). | |