Turn Any URL Into Sales Intelligence.

Welcome to the Prompt Fuel API documentation. Our web scraping API provides a simple, reliable way to extract data from any website with a 99.9% success rate.

Base URL

https://api.promptfuel.io

Key Features

  • Universal anti-bot bypass (Cloudflare, DataDome, PerimeterX, and 50+ others)
  • JavaScript rendering with real Chrome browsers
  • Automatic CAPTCHA solving
  • Premium proxy network across 195+ countries
  • Smart retry logic with exponential backoff
  • Structured data extraction
Response Times: Scraping requests typically take 5-30 seconds depending on the target website's complexity and anti-bot measures. All our code examples use asynchronous patterns to handle these longer response times properly.

Authentication

All API requests require authentication using your API key. You can obtain your API key from your dashboard after signing up.

Authentication Method

Include your API key in the Authorization header with the Bearer scheme:

Authorization Header
Authorization: Bearer YOUR_API_KEY
Security: Never expose your API key in client-side code or public repositories. Always use environment variables or secure key management systems.

Rate Limits

Rate limits depend on your subscription plan:

  • Developer: 25 concurrent requests
  • Startup: 50 concurrent requests
  • Professional: 100 concurrent requests
  • Enterprise: Custom limits

Quick Start

Get started with Prompt Fuel in under 5 minutes. Here's a simple example to scrape your first website:

1. Get Your API Key

Sign up for a free account and get your API key from the dashboard.

2. Make Your First Request

Simple Request
curl https://api.promptfuel.io/scrape/example.com \
  -H "Authorization: Bearer YOUR_API_KEY"

3. Handle the Response

The API returns a JSON response with the scraped data:

{
  "url": "https://example.com",
  "title": "Example Domain",
  "emails": ["info@example.com"],
  "phones": ["+1-234-567-8900"],
  "social_links": {
    "twitter": ["https://twitter.com/example"],
    "linkedin": ["https://linkedin.com/company/example"]
  },
  "tech": {
    "analytics": ["google_analytics"],
    "frameworks": ["wordpress"],
    "cdn": ["cloudflare"]
  },
  "html": "...",
  "status": "success",
  "timestamp": "2024-08-24T12:34:56Z"
}

Python Integration

Use Python with asyncio and aiohttp for efficient asynchronous web scraping with Prompt Fuel.

Installation

Install Dependencies
pip install aiohttp

Basic Example

async_scraper.py
import asyncio
import aiohttp
import json

async def scrape_website(url):
    """Scrape a website using Prompt Fuel API"""
    api_url = f"https://api.promptfuel.io/scrape/{url}"
    headers = {
        "Authorization": "Bearer YOUR_API_KEY"
    }
    
    async with aiohttp.ClientSession() as session:
        try:
            async with session.get(api_url, headers=headers) as response:
                if response.status == 200:
                    data = await response.json()
                    return data
                else:
                    error = await response.text()
                    print(f"Error {response.status}: {error}")
                    return None
        except Exception as e:
            print(f"Request failed: {e}")
            return None

# Example usage
async def main():
    result = await scrape_website("techcrunch.com")
    if result:
        print(json.dumps(result, indent=2))

if __name__ == "__main__":
    asyncio.run(main())
Best Practice: Always use async/await for web scraping to avoid blocking your application. The examples above handle timeouts, retries, and rate limiting automatically.

JavaScript Integration

Use modern JavaScript with async/await to integrate Prompt Fuel API in browser or Node.js environments.

Browser JavaScript

Modern Async/Await
async function scrapeWebsite(domain) {
    const apiKey = 'YOUR_API_KEY';
    const url = `https://api.promptfuel.io/scrape/${domain}`;
    
    try {
        const response = await fetch(url, {
            headers: {
                'Authorization': `Bearer ${apiKey}`
            }
        });
        
        if (!response.ok) {
            throw new Error(`HTTP error! status: ${response.status}`);
        }
        
        const data = await response.json();
        console.log('Scraping successful:', data);
        return data;
        
    } catch (error) {
        console.error('Error:', error);
        throw error;
    }
}

// Example usage
scrapeWebsite('techcrunch.com')
    .then(data => {
        console.log(`Title: ${data.title}`);
        console.log(`Emails found: ${data.emails.join(', ')}`);
    })
    .catch(error => {
        console.error('Error:', error);
    });

Node.js Integration

Build robust server-side applications with Node.js and the Prompt Fuel API.

Installation

Install Dependencies
npm install axios

Express.js API Example

server.js
const express = require('express');
const axios = require('axios');

const app = express();
app.use(express.json());

const PROMPT_FUEL_API_KEY = process.env.PROMPT_FUEL_API_KEY;
const BASE_URL = 'https://api.promptfuel.io';

// Create axios instance with default config
const api = axios.create({
    baseURL: BASE_URL,
    headers: {
        'Authorization': `Bearer ${PROMPT_FUEL_API_KEY}`
    },
    timeout: 60000 // 60 seconds
});

// Scrape endpoint
app.get('/api/scrape/:domain', async (req, res) => {
    const { domain } = req.params;
    
    try {
        console.log(`Starting scrape for ${domain}...`);
        const startTime = Date.now();
        
        const response = await api.get(`/scrape/${domain}`);
        const data = response.data;
        
        console.log(`Scrape completed in ${Date.now() - startTime}ms`);
        
        res.json({
            ...data,
            scrape_time: Date.now() - startTime,
            cached: false
        });
        
    } catch (error) {
        console.error(`Failed to scrape ${domain}:`, error);
        
        res.status(500).json({
            error: 'Scraping failed',
            message: error.message,
            domain
        });
    }
});

// Batch endpoint
app.post('/api/scrape/batch', async (req, res) => {
    const { domains } = req.body;
    
    if (!Array.isArray(domains) || domains.length === 0) {
        return res.status(400).json({
            error: 'Invalid request',
            message: 'domains must be a non-empty array'
        });
    }
    
    const results = await Promise.allSettled(
        domains.map(domain => api.scrape(domain))
    );
    
    const response = results.map((result, index) => ({
        domain: domains[index],
        status: result.status,
        data: result.status === 'fulfilled' ? result.value : null,
        error: result.status === 'rejected' ? result.reason.message : null
    }));
    
    res.json(response);
});

// Health check
app.get('/health', (req, res) => {
    res.json({
        status: 'ok',
        cache_size: cache.size
    });
});

// Start server
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
    console.log(`Server running on port ${PORT}`);
});
Production Tip: Always use environment variables for API keys and implement proper rate limiting in production applications.

Error Codes

Comprehensive list of error codes returned by the Prompt Fuel API to help you debug and handle different scenarios.

Domain and Access Errors (DOM series)

DOM001: Domain Blacklisted
Domain forbidden due to access restrictions
DOM002: Premium Proxy Required
Domain requires proxy due to anti-bot protection
DOM003: Anti-Bot Protection Detected
Domain blocked automated access
DOM004: Domain Not Found
Domain doesn't exist or DNS error
DOM005: Domain Timeout
Domain took too long to respond

Request Errors (REQ series)

REQ001: Invalid Domain Format
Domain not in valid format
REQ002: Missing Required Parameters
Required parameters missing from request
REQ003: Invalid Parameter Values
Parameters have invalid values
REQ004: Request Payload Too Large
Request payload exceeds limits
REQ005: Unsupported Content Type
Content type not supported

Browser and Rendering Errors (BRW series)

BRW001: Browser Initialization Failed
Unable to initialize browser
BRW002: Page Load Timeout
Page took too long to load
BRW003: JavaScript Execution Failed
JS execution error
BRW004: Browser Pool Exhausted
All browser instances in use
BRW005: Page Navigation Failed
Navigation error

Proxy Errors (PRX series)

PRX001: Proxy Connection Failed
Unable to connect through proxy
PRX002: Proxy Authentication Failed
Proxy auth error
PRX003: Proxy Timeout
Proxy connection timed out
PRX004: No Proxy Available
No proxy servers available
PRX005: Proxy Blocked by Target
Target site blocks proxy traffic

Content Extraction Errors (EXT series)

EXT001: No Content Extracted
Page loaded but no content found
EXT002: Content Parsing Failed
Unable to parse page content
EXT003: Invalid Content Format
Content format invalid
EXT004: Content Too Large
Content exceeds size limits
EXT005: Extraction Timeout
Content extraction timed out

System Errors (SYS series)

SYS001: Database Connection Failed
DB connection error
SYS002: Redis Connection Failed
Redis connection error
SYS003: Internal Server Error
Unexpected server error
SYS004: Resource Exhausted
Server resources exhausted
SYS005: Configuration Error
System configuration error

Rate Limiting and Quota Errors (LIM series)

LIM001: Request Rate Exceeded
Request rate limit exceeded
LIM002: Concurrency Limit Reached
Too many concurrent requests
LIM003: Daily Quota Exceeded
Daily usage quota exceeded
LIM004: IP Address Blocked
IP address blocked

Authentication Errors (AUTH series)

AUTH001: API Key Missing
API key not provided
AUTH002: Invalid API Key
API key invalid
AUTH003: API Key Expired
API key expired
AUTH004: Insufficient Permissions
Insufficient API permissions
Tip: All error codes are returned in the response body with additional context and suggested solutions. Use these codes to implement proper error handling in your applications.

cURL Integration

Use cURL for quick testing and command-line integration with the Prompt Fuel API.

Basic Request

Simple GET Request
curl -X GET "https://api.promptfuel.io/scrape/example.com" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

With Custom Options

Advanced cURL Request
curl -X GET "https://api.promptfuel.io/scrape/example.com" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "proxy_type": "premium",
    "country": "US",
    "wait_for": "networkidle",
    "timeout": 30
  }' \
  --max-time 60

Go Integration

Build high-performance applications with Go and the Prompt Fuel API.

Basic Example

main.go
package main

import (
    "encoding/json"
    "fmt"
    "io/ioutil"
    "net/http"
    "time"
)

type ScrapeResult struct {
    URL     string   `json:"url"`
    Title   string   `json:"title"`
    Emails  []string `json:"emails"`
    Status  string   `json:"status"`
}

func scrapeWebsite(domain, apiKey string) (*ScrapeResult, error) {
    url := fmt.Sprintf("https://api.promptfuel.io/scrape/%s", domain)
    
    client := &http.Client{
        Timeout: 60 * time.Second,
    }
    
    req, err := http.NewRequest("GET", url, nil)
    if err != nil {
        return nil, err
    }
    
    req.Header.Set("Authorization", "Bearer "+apiKey)
    req.Header.Set("Content-Type", "application/json")
    
    resp, err := client.Do(req)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()
    
    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return nil, err
    }
    
    var result ScrapeResult
    err = json.Unmarshal(body, &result)
    if err != nil {
        return nil, err
    }
    
    return &result, nil
}

func main() {
    apiKey := "YOUR_API_KEY"
    domain := "example.com"
    
    result, err := scrapeWebsite(domain, apiKey)
    if err != nil {
        fmt.Printf("Error: %v\n", err)
        return
    }
    
    fmt.Printf("Title: %s\n", result.Title)
    fmt.Printf("Emails: %v\n", result.Emails)
}

Rust Integration

Build fast, safe applications with Rust and the Prompt Fuel API.

Installation

Cargo.toml
[dependencies]
tokio = { version = "1.0", features = ["full"] }
reqwest = { version = "0.11", features = ["json"] }
serde = { version = "1.0", features = ["derive"] }

Basic Example

main.rs
use reqwest::Client;
use serde::{Deserialize, Serialize};
use std::time::Duration;

#[derive(Debug, Deserialize)]
struct ScrapeResult {
    url: String,
    title: String,
    emails: Vec,
    status: String,
}

#[tokio::main]
async fn main() -> Result<(), Box> {
    let api_key = "YOUR_API_KEY";
    let domain = "example.com";
    
    let client = Client::builder()
        .timeout(Duration::from_secs(60))
        .build()?;
    
    let url = format!("https://api.promptfuel.io/scrape/{}", domain);
    
    let response = client
        .get(&url)
        .header("Authorization", format!("Bearer {}", api_key))
        .header("Content-Type", "application/json")
        .send()
        .await?;
    
    if response.status().is_success() {
        let result: ScrapeResult = response.json().await?;
        println!("Title: {}", result.title);
        println!("Emails: {:?}", result.emails);
    } else {
        println!("Error: {}", response.status());
    }
    
    Ok(())
}

PHP Integration

Integrate Prompt Fuel API into your PHP applications with cURL or Guzzle.

Basic cURL Example

scraper.php
 $url,
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_TIMEOUT => 60,
        CURLOPT_HTTPHEADER => [
            "Authorization: Bearer " . $apiKey,
            "Content-Type: application/json"
        ]
    ]);
    
    $response = curl_exec($curl);
    $httpCode = curl_getinfo($curl, CURLINFO_HTTP_CODE);
    $error = curl_error($curl);
    
    curl_close($curl);
    
    if ($error) {
        throw new Exception("cURL Error: " . $error);
    }
    
    if ($httpCode !== 200) {
        throw new Exception("HTTP Error: " . $httpCode);
    }
    
    return json_decode($response, true);
}

// Example usage
try {
    $apiKey = "YOUR_API_KEY";
    $domain = "example.com";
    
    $result = scrapeWebsite($domain, $apiKey);
    
    echo "Title: " . $result['title'] . "\n";
    echo "Emails: " . implode(", ", $result['emails']) . "\n";
    
} catch (Exception $e) {
    echo "Error: " . $e->getMessage() . "\n";
}

?>

Ruby Integration

Use Ruby with Net::HTTP or HTTParty for elegant API integration.

Basic Example

scraper.rb
require 'net/http'
require 'json'
require 'uri'

class PromptFuelScraper
  def initialize(api_key)
    @api_key = api_key
    @base_url = 'https://api.promptfuel.io'
  end

  def scrape_website(domain)
    uri = URI("#{@base_url}/scrape/#{domain}")
    
    http = Net::HTTP.new(uri.host, uri.port)
    http.use_ssl = true
    http.read_timeout = 60
    
    request = Net::HTTP::Get.new(uri)
    request['Authorization'] = "Bearer #{@api_key}"
    request['Content-Type'] = 'application/json'
    
    response = http.request(request)
    
    case response.code.to_i
    when 200
      JSON.parse(response.body)
    else
      raise "HTTP Error #{response.code}: #{response.body}"
    end
  rescue StandardError => e
    puts "Error: #{e.message}"
    nil
  end
end

# Example usage
scraper = PromptFuelScraper.new('YOUR_API_KEY')
result = scraper.scrape_website('example.com')

if result
  puts "Title: #{result['title']}"
  puts "Emails: #{result['emails'].join(', ')}"
else
  puts "Scraping failed"
end

Java Integration

Build enterprise applications with Java and the Prompt Fuel API.

Basic Example

PromptFuelScraper.java
import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.JsonNode;

public class PromptFuelScraper {
    private final String apiKey;
    private final HttpClient httpClient;
    private final ObjectMapper objectMapper;
    
    public PromptFuelScraper(String apiKey) {
        this.apiKey = apiKey;
        this.httpClient = HttpClient.newBuilder()
            .connectTimeout(Duration.ofSeconds(10))
            .build();
        this.objectMapper = new ObjectMapper();
    }
    
    public JsonNode scrapeWebsite(String domain) throws IOException, InterruptedException {
        String url = "https://api.promptfuel.io/scrape/" + domain;
        
        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .timeout(Duration.ofSeconds(60))
            .header("Authorization", "Bearer " + apiKey)
            .header("Content-Type", "application/json")
            .GET()
            .build();
        
        HttpResponse response = httpClient.send(request, 
            HttpResponse.BodyHandlers.ofString());
        
        if (response.statusCode() == 200) {
            return objectMapper.readTree(response.body());
        } else {
            throw new RuntimeException("HTTP Error " + response.statusCode() + ": " + response.body());
        }
    }
    
    public static void main(String[] args) {
        try {
            PromptFuelScraper scraper = new PromptFuelScraper("YOUR_API_KEY");
            JsonNode result = scraper.scrapeWebsite("example.com");
            
            System.out.println("Title: " + result.get("title").asText());
            System.out.println("Emails: " + result.get("emails").toString());
            
        } catch (Exception e) {
            System.err.println("Error: " + e.getMessage());
        }
    }
}

C# Integration

Build .NET applications with C# and the Prompt Fuel API.

Basic Example

PromptFuelScraper.cs
using System;
using System.Net.Http;
using System.Threading.Tasks;
using Newtonsoft.Json;
using System.Collections.Generic;

public class ScrapeResult
{
    public string Url { get; set; }
    public string Title { get; set; }
    public List Emails { get; set; }
    public string Status { get; set; }
}

public class PromptFuelScraper
{
    private readonly HttpClient _httpClient;
    private readonly string _apiKey;
    
    public PromptFuelScraper(string apiKey)
    {
        _apiKey = apiKey;
        _httpClient = new HttpClient()
        {
            Timeout = TimeSpan.FromSeconds(60)
        };
        _httpClient.DefaultRequestHeaders.Add("Authorization", $"Bearer {apiKey}");
    }
    
    public async Task ScrapeWebsiteAsync(string domain)
    {
        try
        {
            string url = $"https://api.promptfuel.io/scrape/{domain}";
            
            HttpResponseMessage response = await _httpClient.GetAsync(url);
            
            if (response.IsSuccessStatusCode)
            {
                string content = await response.Content.ReadAsStringAsync();
                return JsonConvert.DeserializeObject(content);
            }
            else
            {
                throw new HttpRequestException($"HTTP Error {response.StatusCode}");
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Error: {ex.Message}");
            return null;
        }
    }
    
    public void Dispose()
    {
        _httpClient?.Dispose();
    }
}

// Example usage
class Program
{
    static async Task Main(string[] args)
    {
        var scraper = new PromptFuelScraper("YOUR_API_KEY");
        
        try
        {
            var result = await scraper.ScrapeWebsiteAsync("example.com");
            
            if (result != null)
            {
                Console.WriteLine($"Title: {result.Title}");
                Console.WriteLine($"Emails: {string.Join(", ", result.Emails)}");
            }
        }
        finally
        {
            scraper.Dispose();
        }
    }
}

Swift Integration

Build iOS and macOS apps with Swift and the Prompt Fuel API.

Basic Example

PromptFuelScraper.swift
import Foundation

struct ScrapeResult: Codable {
    let url: String
    let title: String
    let emails: [String]
    let status: String
}

class PromptFuelScraper {
    private let apiKey: String
    private let session: URLSession
    
    init(apiKey: String) {
        self.apiKey = apiKey
        let config = URLSessionConfiguration.default
        config.timeoutIntervalForRequest = 60.0
        self.session = URLSession(configuration: config)
    }
    
    func scrapeWebsite(domain: String) async throws -> ScrapeResult {
        guard let url = URL(string: "https://api.promptfuel.io/scrape/\(domain)") else {
            throw URLError(.badURL)
        }
        
        var request = URLRequest(url: url)
        request.httpMethod = "GET"
        request.setValue("Bearer \(apiKey)", forHTTPHeaderField: "Authorization")
        request.setValue("application/json", forHTTPHeaderField: "Content-Type")
        
        let (data, response) = try await session.data(for: request)
        
        guard let httpResponse = response as? HTTPURLResponse,
              httpResponse.statusCode == 200 else {
            throw URLError(.badServerResponse)
        }
        
        let decoder = JSONDecoder()
        return try decoder.decode(ScrapeResult.self, from: data)
    }
}

// Example usage
Task {
    let scraper = PromptFuelScraper(apiKey: "YOUR_API_KEY")
    
    do {
        let result = try await scraper.scrapeWebsite(domain: "example.com")
        print("Title: \(result.title)")
        print("Emails: \(result.emails.joined(separator: ", "))")
    } catch {
        print("Error: \(error)")
    }
}

Kotlin Integration

Build Android and server-side applications with Kotlin and the Prompt Fuel API.

Basic Example

PromptFuelScraper.kt
import kotlinx.coroutines.*
import kotlinx.serialization.*
import kotlinx.serialization.json.*
import java.net.http.HttpClient
import java.net.http.HttpRequest
import java.net.http.HttpResponse
import java.net.URI
import java.time.Duration

@Serializable
data class ScrapeResult(
    val url: String,
    val title: String,
    val emails: List,
    val status: String
)

class PromptFuelScraper(private val apiKey: String) {
    private val httpClient = HttpClient.newBuilder()
        .connectTimeout(Duration.ofSeconds(10))
        .build()
    
    private val json = Json { ignoreUnknownKeys = true }
    
    suspend fun scrapeWebsite(domain: String): ScrapeResult = withContext(Dispatchers.IO) {
        val url = "https://api.promptfuel.io/scrape/$domain"
        
        val request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .timeout(Duration.ofSeconds(60))
            .header("Authorization", "Bearer $apiKey")
            .header("Content-Type", "application/json")
            .GET()
            .build()
        
        val response = httpClient.send(request, HttpResponse.BodyHandlers.ofString())
        
        if (response.statusCode() == 200) {
            json.decodeFromString(response.body())
        } else {
            throw Exception("HTTP Error ${response.statusCode()}: ${response.body()}")
        }
    }
}

// Example usage
fun main() = runBlocking {
    val scraper = PromptFuelScraper("YOUR_API_KEY")
    
    try {
        val result = scraper.scrapeWebsite("example.com")
        println("Title: ${result.title}")
        println("Emails: ${result.emails.joinToString(", ")}")
    } catch (e: Exception) {
        println("Error: ${e.message}")
    }
}