The desktop is part of the experiment
A desktop-agent result depends on more than the model. The operating system, application versions, display geometry, files, account state, and action controller shape the task. A score obtained in one prepared environment does not automatically describe performance on a personal workstation with different applications and permissions.
OSWorld V2 provides a concrete example of environment versioning: its releases coordinate code, tasks, assets, mocked websites, and provider images. Preserve these identifiers when recording a result. “Desktop benchmark” is a category label, not enough detail to reproduce or fairly compare an experiment.
Check the outcome behind the screen
Choose a result whose correctness can be inspected independently: the requested file was created, its content matches the task, and it is stored at the approved location. A screenshot showing a completed-looking application is useful evidence, but it may not prove the file was saved or that the intended record was changed.
Our editorial evaluation sheet separates navigation progress, final artifact validity, unintended side effects, intervention, and elapsed time. Include tasks with modal dialogs, multiple windows, unsaved changes, and application errors. These expose different failure modes from a browser page that has one obvious button.
Keep control permissions consistent
A system with shell access, a filesystem API, and an application-specific tool may solve a task differently from a screenshot-only system. Both results can be useful, but the allowed tool surface must be visible. Do not attribute the outcome solely to visual reasoning when a privileged tool performed the decisive step.
Likewise, keep the starting permissions and account scope fixed. A run that skips login through a prepared session is not directly comparable with one that must authenticate. Record human interventions and whether they are allowed by the benchmark protocol. A hidden rescue can turn an apparently autonomous success into a different measurement.
Translate the evidence into a pilot
Use public desktop results to choose what to investigate, then reproduce a small set of your actual application tasks in a dedicated test environment. Set an explicit stop boundary for deletion, transmission, and irreversible changes. Validate the result with code or a reviewer who can inspect the native artifact.
Measure both accepted outputs and safe stopping behavior. A system that declines an ambiguous destructive action may be preferable for a workflow even if a broad completion metric does not reward that decision. COMPUTER USE does not claim to have rerun the cited desktop benchmarks; this guide explains the evidence needed to interpret them.