Computer use for portal automation
Build one adapter per portal and keep business rules above the interface layer.
Define the job
Execute a defined read or draft-preparation task in an authorized third-party portal. Normalize outputs into a shared schema while preserving portal-specific labels, identifiers, and evidence for each record.
Should this use computer use at all?
Ask whether the portal provides a supported API, export, or partner integration. If it does, compare its coverage before assuming the visible interface is the only path.
API vs. computer use →Architecture options
Known portal adapter
Use a deterministic controller for a stable navigation path. Each adapter returns explicit states such as completed, missing record, reauthentication required, and review needed.
Agent-assisted variation
Evaluate an agent for a tightly scoped navigation problem when the portal layout varies. Keep account scope, output schema, and consequential-action checks outside the model prompt.
Failure modes & approval
Where the workflow can break
- An interface redesign can make a formerly correct selector point at the wrong action.
- Portal status labels may differ from the internal business taxonomy.
- A retry after a partially completed job can duplicate uploads or submissions.
Human approval point
Authorize the portal and record scope in advance, then review any consequential submission or newly discovered action. An inaccessible record should produce an exception rather than a broader search across accounts.
Design the handoff →Tools to evaluate
Playwright
Deterministic browser automation and testing across Chromium, Firefox, and WebKit.
Verified Sep 27, 2026Control frameworkSelenium
A browser automation ecosystem built around WebDriver, IDE, and distributed Grid execution.
Verified Sep 27, 2026Browser infrastructureBrowserbase
Managed browser sessions with observability, proxies, and computer-use integrations.
Verified Sep 27, 2026Agent & orchestrationBrowser Use
An open-source browser agent library with managed agents and cloud browser sessions.
Verified Sep 27, 2026Agent & orchestrationSkyvern
Vision-assisted browser workflows with a developer SDK and a visual workflow builder.
Verified Sep 27, 2026Budget the whole workflow
- Number of portal-specific adapters
- Session and layout change rate
- Exception handling and monitoring
Implementation playbooks
Download invoices from vendor portals
Produce a verified invoice bundle with account, period, and duplicate checks.
Check claim status across authorized portals
Collect status evidence and route ambiguous identities or meanings to review.
Fill a form with an explicit approval boundary
Prepare the exact payload, pause at the consequential step, and verify the committed result.
Sources & verification
Reviewed Sep 27, 2026. Architecture recommendations are editorial analysis; linked vendor documentation supports the underlying capability and safety facts.
- playwright.dev/docs/locators
- www.skyvern.com/docs/developers/getting-started/quickstart
- developers.openai.com/api/docs/guides/tools-computer-use