Bright Data - Web Scraping & Search

Direct API access to Bright Data's Web Unlocker and SERP APIs.

Setup

1. Get your API Key:
Get a key from [Bright Data Dashboard]().

2. Create a Web Unlocker zone:
Create a zone at brightdata.com/cp by clicking "Add" (top-right), selecting "Unlocker zone".

3. Set environment variables:

export BRIGHTDATA_API_KEY="your-api-key"
export BRIGHTDATA_UNLOCKER_ZONE="your-zone-name"

Usage

Google Search

Search Google and get structured JSON results (title, link, description).

bash scripts/search.sh "query" [cursor]

• cursor: Optional page number for pagination (0-indexed, default: 0)

Web Scraping

Scrape any webpage as markdown. Bypasses bot detection and CAPTCHA.

bash scripts/scrape.sh "url"

Output Formats

Search Results

Returns JSON with structured organic array:

{
  "organic": [
    {"link": "...", "title": "...", "description": "..."}
  ]
}

Scrape Results

Returns clean markdown content from the webpage.