Twitter Search and Analysis

Overview

Search Twitter for keywords using advanced search syntax, fetch up to 1000 relevant tweets, and analyze the data to produce professional reports with insights, statistics, and actionable recommendations.

Prerequisites

API Key Required: Users must configure their Twitter API key from

The API key can be provided in three ways:

Environment variable (recommended): Set TWITTER_API_KEY in your ~/.bashrc or ~/.zshrc

echo 'export TWITTER_API_KEY="your_key_here"' >> ~/.bashrc
   source ~/.bashrc

As an argument: Use --api-key YOUR_KEY with the wrapper script

Passed directly: As first argument to the Python script

Quick Start

Using the Wrapper Script (Recommended)

The wrapper script automatically handles environment variable loading and dependency checks:

# Basic search (uses TWITTER_API_KEY from shell config)
./scripts/run_search.sh "AI"

# With custom API key
./scripts/run_search.sh "AI" --api-key YOUR_KEY

# With options
./scripts/run_search.sh "\"Claude AI\"" --max-results 100 --format summary

# Advanced query
./scripts/run_search.sh "from:elonmusk since:2024-01-01" --query-type Latest

Direct Python Script Usage

# Search for a keyword
scripts/twitter_search.py "$API_KEY" "AI"

# Search with multiple keywords
scripts/twitter_search.py "$API_KEY" "\"ChatGPT\" OR \"Claude AI\""

# Search from specific user
scripts/twitter_search.py "$API_KEY" "from:elonmusk"

# Search with date range
scripts/twitter_search.py "$API_KEY" "Bitcoin since:2024-01-01"

Advanced Queries

# Complex query: AI tweets from verified users, English only
scripts/twitter_search.py "$API_KEY" "AI OR \"machine learning\" lang:en filter:verified"

# Recent crypto tweets with minimum engagement
scripts/twitter_search.py "$API_KEY" "Bitcoin min_retweets:10 lang:en"

# From specific influencers
scripts/twitter_search.py "$API_KEY" "from:elonmusk OR from:VitalikButerin since:2024-01-01"

Output Format

# Full JSON with all tweets
scripts/twitter_search.py "$API_KEY" "AI" --format json

# Summary with statistics (default)
scripts/twitter_search.py "$API_KEY" "AI" --format summary

Options

• --max-results N: Maximum tweets to fetch (default: 1000)
• --query-type Latest|Top: Sort order (default: Top for relevance)
• --format json|summary: Output format (default: summary)

Workflow

1. Understand User Requirements

Clarify the analysis goal:

• What topic/keyword to search?


• Date range preference?


• Specific users to include/exclude?


• Language preference?


• Type of insights needed (trends, sentiment, influencers)?

2. Build the Search Query

Use [Twitter Advanced Search]() syntax:

Syntax

Example

Description

keyword	AI	Single keyword
"phrase"	"machine learning"	Exact phrase
OR	AI OR ChatGPT	Either term
from:user	from:elonmusk	From specific user
to:user	to:elonmusk	Reply to user
since:DATE	since:2024-01-01	After date
until:DATE	until:2024-12-31	Before date
lang:xx	lang:en	Language code
#hashtag	#AI	Hashtag
filter:links	filter:links	Tweets with links
min_retweets:N	min_retweets:100	Minimum retweets

3. Fetch Data

Execute the search script:

scripts/twitter_search.py "$API_KEY" "YOUR_QUERY" --max-results 1000 --query-type Top

Important: Default is 1000 tweets maximum. The script automatically:

• Paginates through all available results


• Stops at 1000 tweets (API limit consideration)


• Handles errors gracefully

4. Analyze and Generate Report

After fetching data, produce a comprehensive professional report with:

Report Structure

Executive Summary (2-3 sentences)

- What was searched
- Key findings overview

Data Overview

- Total tweets analyzed
- Date range of data
- Query parameters used

Key Metrics

- Total engagement (likes, retweets, replies, quotes, views)
- Average engagement per tweet
- Language distribution
- Reply vs. original tweet ratio

Top Content Analysis

- Most retweeted tweets (with URL links to original tweets)
- Most liked tweets (with URL links to original tweets)
- Top hashtags with frequency
- Most mentioned users
- Selected tweet examples with full URL references

Influencer Analysis

- Top users by follower count
- Most active users
- Verified user percentage

Trend Insights (based on data patterns)

- Emerging themes
- Sentiment indicators
- Temporal patterns
- Conversation drivers

Key Takeaways

- 3-5 bullet points of core insights
- Data-backed conclusions

Actionable Recommendations

- Specific, implementable suggestions
- Based on the data findings
- Prioritized by impact

Analysis Guidelines

• Be data-driven: Every claim should reference actual metrics
• Provide context: Explain why metrics matter
• Identify patterns: Look for trends across the dataset
• Stay objective: Present facts, avoid speculation
• Be specific: Recommendations should be concrete and actionable
• Consider external context: Use web search for background when relevant

5. Output Format

Present the report in clear markdown with:

• Headers for each section


• Tables for structured data


• Bullet points for lists


• Bold for key metrics


• Code blocks for tweet examples


• Clickable URLs for all referenced tweets (format: [@username]())

Tweet URL Format

Always include clickable links to tweets:

| Author | Tweet | URL |
|--------|-------|-----|
| @user | Summary of tweet content | [View]() |

Or inline format:

- **@username**: Tweet summary - [View Tweet]()

Query Examples by Use Case

Trend Analysis

"AI" OR "artificial intelligence" lang:en min_retweets:50

Competitor Monitoring

from:competitor1 OR from:competitor2 since:2024-01-01

Product Launch Tracking

#ProductName OR "Product Name" lang:en filter:verified

Crisis Monitoring

#BrandName OR "Brand Name" lang:en --query-type Latest

Influencer Discovery

#Topic lang:en min_retweets:100 min_faves:500

Sentiment Analysis

"brand name" OR #BrandName lang:en --max-results 1000

Resources

scripts/run_search.sh (Wrapper Script)

Convenience wrapper that handles environment variable loading and dependency checks:

• Automatically loads TWITTER_API_KEY from ~/.bashrc or ~/.zshrc


• Checks Python availability and installs missing dependencies


• Provides user-friendly error messages


• Supports all command-line options from the Python script

Usage:

./scripts/run_search.sh <query> [options]

Options:

• --api-key KEY: Override environment variable API key


• --max-results N: Maximum tweets to fetch (default: 1000)


• --query-type Latest|Top: Sort order (default: Top)


• --format json|summary: Output format (default: json)

scripts/twitter_search.py

Executable Python script that:

• Fetches tweets from Twitter API


• Handles pagination automatically


• Extracts key tweet metrics


• Calculates aggregate statistics


• Outputs structured JSON data

Usage:

scripts/twitter_search.py <api_key> <query> [options]

references/twitter_api.md

Comprehensive API documentation including:

• Complete parameter reference


• Query syntax guide


• Response structure details


• Pagination instructions


• Best practices for analysis


• Error handling guide

Read this when: Building complex queries or understanding data structure.

Tips for Better Analysis

Use Top query type for trend analysis (more relevant results)

Set date filters for timely insights

Filter by language for accurate text analysis

Include minimum engagement to filter noise

Combine with web search to validate trends

Look beyond metrics - analyze content themes

Track hashtags to identify sub-conversations

Identify influencers by combining followers + engagement

Error Handling

If the script fails:

• Check API key validity


• Verify query syntax


• Ensure network connectivity


• Check rate limits (if applicable)


• Review error messages for specific issues