SkillsBench: Benchmarking How Well Skills Work Across Diverse Tasks
The first evaluation framework that measures how skills work, and the first dataset that measures how powerful models are at using skills on expert-curated tasks across high-GDP-value, diverse domains.
How SkillsBench Works
SkillsBench evaluates AI agents across three abstraction layers, mirroring how traditional computing systems are structured.
Skills Layer
Domain-specific capabilities and workflows that extend agent functionality. Like applications on an OS, skills provide specialized knowledge and tools for particular tasks.
Agent Harness Layer
The execution environment that orchestrates agents, manages tool access, and handles I/O. Analogous to an operating system that mediates between applications and hardware.
Models Layer
The foundational AI models that power reasoning and generation. Like CPUs, they provide the raw computational capability that upper layers build upon.
Agent Performance
Pass rates across 16 agent–model configurations on SkillsBench (94 tasks, up to 3 trials per task).
| # | Agent | With Skills |
|---|---|---|
| 1 | OpenHandsGPT-5.5 | 55.0% |
| 2 | OpenHandsOpus 4.7 | 51.4% |
| 3 | Gemini CLIGemini 3.1 Pro | 49.1% |
| 4 | OpenHandsGemini 3.1 Pro | 48.9% |
| 5 | CodexGPT-5.5 | 48.6% |
| 6 | Claude CodeOpus 4.7 | 46.1% |
| 7 | OpenHandsDeepSeek V4 Pro | 39.7% |
| 8 | OpenHandsKimi K2.6 | 34.6% |
| 9 | OpenHandsGLM 5.1 | 34.5% |
| 10 | OpenHandsDeepSeek V4 Flash | 31.5% |
| 11 | OpenHandsGPT-5.4 Mini | 26.1% |
| 12 | OpenHandsMiniMax M2.7 | 26.0% |
| 13 | OpenHandsSonnet 4.6 | 25.9% |
| 14 | OpenHandsQwen3.6 Plus | 20.7% |
| 15 | OpenHandsGrok 4.3 | 19.7% |
| 16 | OpenHandsGemini 3.1 Flash Lite | 12.4% |
Task Registry
You need to calculate the mass of a 3D printed part. The input (`/root/scan_data.stl`) is a binary STL, but the 2-byte "Attribute Byte Count" at the end of each triangle record is being used to store the **Material ID** of the object.
You need to implement an Adaptive Cruise Control (ACC) simulation that maintains the set speed (30m/s) when no vehicles are detected ahead, and automatically adjusts speed to maintain a safe following distance when a vehicle is detected ahead. The targets are: speed rise time <10s, speed overshoot <
Detect BGP route oscillation and leaks in an Azure Virtual WAN network topology and evaluate which solutions can fix them.
As an optimization and mathematical modeling expert in a shared bike service company, every night, you need to rebalance the bikes among the stations in your responsible region. So the basic information is that you can control several rebalancing vehicles (`vehicle_count`). The vehicles can start dr
You are helping a research team verify the integrity of their bibliography before submitting a paper. The team suspects that some citations in their BibTeX file may be fake or hallucinated.
# Sid Meier's Civilization 6 District Adjacency Bonus Optimizer
Fill the California Small Claims Court form at `/root/sc100-blank.pdf` based on the case description below, and save the filled one to `/root/sc100-filled.pdf`. Only fill in the necessary fields and leave the court-filled, optional fields or fields not mentioned in the case description below empty.
You are a materials science researcher. You want to analyze the crystal structure of a few samples. You are given a few CIF files generated by SHELX through X-ray Diffraction.
You’re given `packets.pcap` (subset of DAPT2020 traffic). Compute the stats and fill in only the `value` column in `/root/network_stats.csv`. Lines starting with `#` are comments—leave them.
Please use D3.js (v6) to visualize input data stored at `/root/data/stock-descriptions.csv` and `/root/data/indiv-stock/` for company stock details and individual stock price histories. Please return the output as a single-page web app at `/root/output/index.html`. I should be able to open it in a w
You will implement a dialogue parser that converts a given text file into a structured JSON graph. You will be given a text file `/app/script.txt`, and output a validated JSON graph `/app/dialogue.json` and visualization `/app/dialogue.dot`. You should implement a function `def parse_script(text: st
Given the system parameters of a drone, generate time-parameterized piecewise continuous trajectories and feedback control design to enable the drone to fly along a pre-defined path in simulation according to a natural language command.
Given a video file /root/input.mp4, help me to analyze the camera motion (egomotion) and detect dynamic objects in this video. You need to give me 2 files: 1. A json file named /root/pred_instructions.json, it should record the mapping frame intervals to motion labels (valid labels: Stay, Dolly In,
You have some earthquake traces stored at `/root/data/wave.mseed`, and station information stored at `/root/data/stations.csv`.
You are a geospatial analyst with expertise in plate tectonics and earthquake analysis. I want to find the earthquake that occurred furthest from the Pacific plate boundary within the Pacific plate itself. Use GeoPandas projections. Output the result to `/root/answer.json` as a JSON file with the fo
In macroeconomics, understanding business cycle dynamics requires separating the trend component (long-term growth) from the cyclical component (short-term fluctuations) of economic time series. This task involves applying standard detrending techniques to analyze how consumption and investment move
You are given a `PDF` file at `/root/input/input.pdf` and a text file at `/root/input/input.txt` containing natural language instructions and updated information. Your task is to update this PDF using the latest information provided in the text file and follow all the instructions in it.
As an operator-planner in an Independent System Operator, you create a base case for tomorrow's peak hour each morning in order to establish the voltage profile and verify that the day-ahead market schedule is AC-feasible. In order to do this, you should use the model described in `math-model.md` to