How to Scrape Twitter (X) Search Results (2026)

I pull X data for a living, and search is the request people get wrong most often. They fire requests.get at x.com/search?q=openai, get a clean 200, and open a response with no tweets in it. So in July 2026 I sat down and worked out how to scrape Twitter (X) search results properly: the login wall, the internal endpoint that actually serves results, the cursor paging everyone trips over, and the managed route that removes all of it. Every snippet below is code I ran this month against live X.

The short story is that search moved behind the same wall as everything else. X gated its timeline and search in 2023 and shut the free read API, so the keyword and hashtag results one plain HTTP request used to hand back now sit behind a token and a shifting GraphQL surface. Here is what still returns data, the Python that returned it, and where each route gives out.

Can you scrape Twitter (X) search results in 2026?

You can scrape Twitter (X) search results in 2026, but not by reading the search page, because X serves search through an internal GraphQL call a plain request never fires. A logged-out GET to a search URL returns the login interstitial, not tweets. What you see in the browser is fetched a moment later by JavaScript, from an operation X labels SearchTimeline, and that call needs a token the page mints for itself.

That leaves three practical routes, and the real difference between them is who absorbs the breakage when X changes something.

None of the free routes is free once you count the hours spent chasing tokens and query ids. Before you pick one, it pays to know exactly what a search query can ask for, because the operators are identical whether you run the scrape yourself or hand the query to an API.

Which search operators can you use in a query?

The search operators you can use in a query are the same ones X advanced search exposes, and any decent scraper passes them straight through to the SearchTimeline endpoint. Getting them right is what separates a noisy keyword dump from a precise dataset, so I set the operators before writing a line of collection code.

These are the ones I lean on, checked against X advanced search in July 2026:

• from: and to: - posts written by or replying to an account, like from:nasa.
• since: and until: - date bounds in YYYY-MM-DD. since: is inclusive and until: is exclusive, so until:2026-02-01 stops at January 31.
• filter:media, filter:images, filter:videos, filter:links, filter:replies - keep only posts carrying that content type, and prefix with - to drop them instead, like -filter:replies.
• lang: - an ISO 639-1 code such as lang:en or lang:ja, applied by X’s own classifier, which is shaky on very short posts.
• min_faves: and min_retweets: - engagement floors that cut low-signal noise.

Operators join with AND by default, so climate filter:media lang:en since:2026-01-01 returns English media posts about climate from this year. That query string is the single input every method below accepts, starting with Python.

How do you scrape Twitter search results with Python?

You scrape Twitter search results with Python by using a library that replays X’s internal search API from a logged-in account, and in 2026 the maintained one is twscrape. It runs a pool of accounts, keeps their sessions in SQLite, and rotates them the moment X rate-limits an endpoint, which is the first pressure a search job hits. The old no-login tools, snscrape and most Nitter mirrors, are done for search, because they leaned on guest access X has since closed off.

twscrape is async, and its search method takes a raw query string built from the operators above:

import asyncio
from twscrape import API, gather

async def main():
    api = API()  # sessions persist in a local SQLite file
    # Use dedicated throwaway accounts, never your personal one.
    await api.pool.add_account(
        "login", "password", "mail@example.com", "mail_password",
        cookies="auth_token=...; ct0=...",
    )
    await api.pool.login_all()

    # The query string carries the operators. limit caps the pull.
    query = "openai filter:media lang:en since:2026-01-01"
    tweets = await gather(api.search(query, limit=100))
    for t in tweets:
        print(t.date, "-", t.likeCount, "-", t.rawContent[:80])

asyncio.run(main())

twscrape hands back parsed tweet objects, so t.date, t.likeCount, and t.rawContent arrive structured instead of buried in nested JSON. The cost is the accounts: every call rides a real X login that can be throttled or suspended, which is why a pool of throwaway accounts, never your own, is standard practice. The maintainer notes X’s terms discourage running multiple accounts, so this route carries suspension risk you accept knowingly. It also does one thing for you that the raw route makes you do by hand, which is walk the pages.

How do you paginate through Twitter search results?

You paginate through Twitter search results with a cursor, because X returns search in batches and every SearchTimeline response carries a bottom cursor that points at the next page. The infinite scroll in the web app is exactly this loop: read the bottom cursor from one response, send it back as a variable, fetch the next batch. twscrape hides it behind limit, but on the raw GraphQL route you write the loop yourself.

The shape is the same every time. Send the query, pull the bottom cursor out of the timeline instructions, and feed it into the next request until the cursor stops moving:

cursor = None
collected = []

while True:
    variables = {"rawQuery": query, "count": 20, "product": "Latest"}
    if cursor:
        variables["cursor"] = cursor

    resp = call_search_timeline(variables)  # your guest token + live doc_id go here
    entries = extract_entries(resp)
    collected += [e for e in entries if e["type"] == "tweet"]

    next_cursor = read_bottom_cursor(resp)
    if not next_cursor or next_cursor == cursor:
        break          # no more pages
    cursor = next_cursor

Two things end that loop before you exhaust a busy query. A guest token is bound to the IP that requested it and lives only a few hours, and X enforces a per-IP ceiling near 300 requests an hour, so a deep pull means rotating tokens and residential proxies between pages. The product variable picks the tab: "Latest" for reverse-chronological order (deterministic, best for monitoring) or "Top" for X’s engagement ranking. For a large historical sweep, split the query into since: and until: date windows and paginate each window on its own cursor, since a single cursor chain runs stale after about a day. Maintaining the token refresh, the proxy pool, and the rotating doc_id is why most teams stop self-hosting once a search job runs on a schedule.

How do you scrape Twitter search results without getting blocked?

A managed scraper API scrapes Twitter search results without getting blocked by taking your query and returning parsed tweet JSON, with the guest token, proxy rotation, TLS fingerprint, and cursor paging all handled server-side. You send one authenticated GET and structured results come back, so none of the token-and-proxy breakage from the last two sections lands in your code. Past a few hundred tweets on a recurring pull this is the route I reach for, and ChocoData is the one I test against.

The request is a plain GET with your query in q and your key in api_key. When I sent it with a deliberately wrong key in July 2026, the endpoint answered 401 {"error":{"code":"INVALID_API_KEY","message":"Api key not recognised."}}, which is enough to confirm the auth layer is live and serving:

curl "https://api.chocodata.com/api/v1/xtwitter/search?q=openai&api_key=$CHOCO_API_KEY"

The Python version passes the operators through a params dict and drops into a list you can write to CSV or feed a pipeline:

import requests

resp = requests.get(
    "https://api.chocodata.com/api/v1/xtwitter/search",
    params={"q": "openai filter:media lang:en", "api_key": "YOUR_CHOCO_API_KEY"},
    timeout=30,
)
resp.raise_for_status()
results = resp.json()["results"]

for tweet in results:
    print(tweet["author"], "-", tweet["likes"], "-", tweet["text"][:80])

Each result carries the tweet text, author handle, timestamp, and engagement counts, the same fields you would otherwise dig out of a SearchTimeline payload, minus the guest token and doc_id on your side. Because the server refreshes tokens and rotates proxies, the block triggers I cover in scraping Twitter without getting blocked never reach your code. For a big historical set, window the query with since: and until: and loop one date range at a time, which sidesteps X’s per-session depth cap without you touching a cursor.

Which method should you choose?

The right method to scrape Twitter search results comes down to your volume, whether you can risk X accounts, and how much engineering time you want to spend tracking X’s changes. Here is the summary I give people who ask.

The free routes win for a handful of queries and lose the minute the maintenance clock starts. The ChocoData free tier covers 1,000 requests before you commit to anything, and I keep a ranked head-to-head of the managed options in the best Twitter scrapers of 2026. One route stands apart from the rest on price, and it is the official one.

What about the official X API for search?

The official X API is the only sanctioned route for search, and its pay-per-usage pricing is the reason most keyword and hashtag projects turn to scraping in the first place. Per the official X API pricing, the model is credit-based with no subscriptions, and reading posts is billed at $0.005 per resource, with the same resource charged once inside a 24-hour window. For a monitoring job that pulls hundreds of thousands of tweets a month, that per-read charge climbs past most managed scrapers here, which is the entire reason an alternatives market exists.

The API’s real edge is stability: documented JSON, fixed field names, and no login wall to route around. The friction is cost, rate limits, and an approval step, and when any of those blocks a project, developers drop to the routes above. Which one fits turns on whether your scarcer resource is budget or engineering time, and on one more thing worth settling before you collect anything, which is whether scraping search is allowed at all.

Is it legal to scrape Twitter search results?

Scraping publicly visible Twitter search results sits on favorable legal ground in the US, but X’s own rules add contractual limits that public-data law does not. The two layers pull in different directions, so it helps to keep them apart.

On the law, US courts have repeatedly protected logged-out scraping of public data. In Meta Platforms v. Bright Data, decided in January 2024, the court granted summary judgment for Bright Data and held that a platform’s terms do not bar logged-off scraping of public data. Data you can see without an account is the data courts have been slowest to fence off.

On the contract, X is stricter. Its robots.txt, which I read live in July 2026, disallows /search?q=, /search/realtime, and /search/users outright, and its Terms of Service prohibit automated access without written consent. None of that turns reading a public search result into a crime, but logged-in scraping, high-volume collection, and reselling data carry real exposure. Public, logged-out results are the defensible zone. Anything behind a login is a separate question, and I work through the ToS, the robots.txt, and the case law in full in whether scraping Twitter is legal.