Where does BenchGecko get its data?

Model catalog and prices from the OpenRouter API, benchmark scores from Epoch AI and public leaderboards (Artificial Analysis, LMArena, LiveBench, Aider, SWE-bench, SEAL, HELM, OpenCompass and others), MCP servers from the MCP Registry, Mindshare from Hacker News, Wikipedia and GitHub, public company figures from SEC filings and market close, GPU rental prices from the Vast.ai marketplace, and a curated infrastructure dataset. Each figure on the site shows its date and source.

How are AI models ranked on BenchGecko?

With the BenchGecko Score: each benchmark score is rescaled from 0 to 100 between the lowest and highest score among tracked models, then averaged. Arena Elo ratings and Artificial Analysis composite indices are excluded, and a model needs at least 3 benchmarks to get a score.

How often is BenchGecko data updated?

Daily: model catalog and OpenRouter prices, per-provider offers, benchmark scores from public leaderboards, MCP servers, Mindshare signals (Hacker News, Wikipedia, GitHub), public company figures (SEC filings and market close), GPU rental prices and Gecko Tests. Twice weekly: private company figures, published only when a trusted source page confirms the number. Weekly: the data digest on the blog. The infrastructure dataset is curated and shows its as-of date. Every figure shows its as-of date.

How does BenchGecko monitor AI API uptime?

BenchGecko calls each tracked provider's public API endpoint every 15 minutes and records the HTTP status and response time. Three failed checks in a row open an incident and two successful checks in a row close it.

Methodology

How we source data, how we score it, and how often each part refreshes.

Tracked today: 1,235 models · 273 providers · 158 benchmarks · 4,923 MCP servers · model data as of Oct 5, 2026

Data Sources

OpenRouter API

Daily

Model catalog and list prices (9 fields: input, output, cache read/write, reasoning, image, audio, web search, per request), context windows, modalities, supported parameters, deprecation dates. Per-provider offers from the endpoints API.

Source: https://openrouter.ai/api/v1/modelsLicense: Public API, no authentication required

Epoch AI

Daily

Benchmark scores and model metadata such as training compute, release dates and organizations.

Source: https://epoch.ai/dataLicense: CC BY 4.0

Public leaderboards

Daily

Artificial Analysis, LMArena, LiveBench, Aider, SWE-bench, Scale SEAL, HELM, OpenCompass, and CJK and multilingual suites. Scores are fetched from each leaderboard and range-checked before they are written.

License: Each leaderboard's own terms

Hugging Face Open LLM Leaderboard

Snapshot

Retired by Hugging Face. BenchGecko keeps its last snapshot; it is no longer fetched.

Source: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboardLicense: Open data

MCP Registry

Daily

MCP server listings with package info, transport types, and repository links.

Source: https://registry.modelcontextprotocol.ioLicense: Open API

Hacker News, Wikipedia, GitHub

Daily

Mindshare signals: daily Hacker News mentions (Algolia API), Wikipedia page views (Wikimedia API) and GitHub stars for every tracked model, lab, agent, person and topic.

Source: https://hn.algolia.com/apiLicense: Public APIs

SEC EDGAR and market close

Daily

Public company figures from filings and the daily market close. Private company figures are researched twice a week and written only when a trusted source page contains the number.

Source: https://www.sec.gov/edgarLicense: Public records

Vast.ai marketplace

Daily

GPU rental prices: median, 10th percentile and minimum on-demand price per GPU-hour for tracked AI chips.

Source: https://vast.aiLicense: Public marketplace data

Hugging Face Hub

Snapshot

Model metadata for open models: parameter counts, licenses, architectures, downloads.

Source: https://huggingface.co/api/modelsLicense: Public API, no authentication required

Infrastructure dataset

Curated · dated

Chips, memory, foundries, systems and energy, curated from datasheets, earnings calls, filings and utility disclosures. Every record carries an as-of date and a source link.

License: Sources linked per record

Gecko Tests

Daily

BenchGecko's own automated tests on the newest models: identity, world map, refusals, knowledge cutoff, tokenizer cost, provider quality, drift. Raw answers are published.

License: CC BY 4.0

Benchmark Scores

We display scores exactly as reported by their source. Scores are not adjusted, weighted, or normalized for cross-benchmark comparison unless explicitly noted. Each score links to its source when available.

When several sources report the same benchmark for a model, BenchGecko keeps the score from the more authoritative source (the benchmark's own leaderboard before aggregators). Scores from official model announcements are added when the public leaderboards do not include them yet.

BenchGecko Score

The BenchGecko Score ranks models on one scale. Each benchmark score is rescaled from 0 to 100 between the lowest and highest score any tracked model has on that benchmark, then the rescaled scores are averaged. Arena Elo ratings and Artificial Analysis composite indices are left out, and a model needs at least 3 benchmarks to get a score. It is a summary signal: check coverage on the model page before drawing strong conclusions.

Pricing

Model list prices come from OpenRouter's public catalog and are shown per 1 million tokens. We track 9 price fields: input, output, cache read, cache write, image, audio, internal reasoning, web search, and per-request fees. Per-provider offers (price, quantization, context window, uptime) come from OpenRouter's endpoints API, so the same model can be compared across hosts.

Prices are checked every day and stored as a daily price history, charted on each model page. Price changes that hold are recorded as dated events in the changelog.

Uptime Monitoring

Provider uptime is measured by calling each tracked provider's public API endpoint every 15 minutes and recording the HTTP status and response time. Three failed checks in a row open an incident; two successful checks in a row close it.

Uptime percentages are computed from the stored checks, which are kept for 35 days. They reflect API endpoint availability, not the availability of individual models.

Open Source Classification

A model is marked "Open Source" if its weights are publicly available for download and use under an OSI-approved license (Apache 2.0, MIT) or a permissive community license (Llama Community License). Models with "open weights" under restrictive licenses are marked separately.

Update Frequency

Data Type	Frequency
API Pricing	Daily
Model Directory	Daily (from OpenRouter)
Benchmark scores	Daily
Provider Uptime	Every 15 minutes
MCP Servers	Daily
Mindshare (Hacker News, Wikipedia, GitHub)	Daily
Company figures	Public: daily · private: twice weekly
GPU rental prices	Daily
Gecko Tests	Daily lab · each model once per test version
Chips, memory, foundries, systems, energy	Curated dataset · as of Apr 14, 2026

Corrections

If you spot an error in our data, email hello@benchgecko.ai or reach out on X @BenchGecko. Every figure keeps its source and date, so corrections can be checked against the original.

Built by the BenchGecko team from public sources. Every figure shows its date and source.