beipo-ai/NorthCUA Open-sourcing the desktop/web computer-use Harness: screenshots and keyboard/mouse interactions, long-form notes, recoverable batches, independent evaluations; model-side integration supported beipo-ai/NorthCUA-Pro(Approx. 397B/17B active, Apache-2.0) or start with the official no-key local tutorial. Let me get to the conclusion first:NorthCUA is suitable for "auditable open-source computer-use runtimes"; weights and Harness must be used in pairs, and GUI coordinates must strictly follow the normalized_1000 system—do not force pixel coordinates into it.. The following is organized based on the publicly available README/model card asDocument Verification VersionGetting Started Guide; please refer to the README file available on the day you access this page for the latest commands. This articleNot attachedLog of a successful run on this machine.
Positioning: What the Open-Source Computer-Use Harness Solves—and What It Doesn't Solve
Harness is not positioned as “just another model that clicks a mouse,” but rather as a tool that strings together observations, actions, work notes, and result verification into a replayable trail. The model handles reasoning and tool invocation; Harness handles execution and pixel-level implementation.
Suitable for:I want to run desktop or web applications in a local/controlled environment; I need a full trace; and I am willing to configure the policy according to the official YAML.
Not suitable for:Just wanting to say "Click this for me" in a chat without a runtime; stuffing Pro weights into an unrelated Agent loop and mixing pixel coordinates; granting write permissions directly to production servers without any isolation.
Project Card
| 项 | Content |
|---|---|
| Harness Repository | https://github.com/beipo-ai/NorthCUA |
| Weights (Example) | NorthCUA-Pro; there are also other series of cards, such as the Nano and Mini |
| License | Apache-2.0 (subject to the repository/model card on the day in question) |
| Coordinate Contract | normalized_1000: x/y ∈ [0,1000], Harness pixel conversion |
| Suitable for | Open-source computer-use, replayable batches, model-agnostic integration |
| Not suitable | Unisolated production machines are malfunctioning; while building a custom workflow, the official tool protocol was lost. |
Use Case Design: normalized_1000 + Official Harness Read-Only Desktop Task
Use Cases:In the app's YAML configuration desktop.coordinate_policy.input_space: normalized_1000, run one using the official HarnessRead-onlyDesktop tasks (such as opening the calculator and observing only the interface, or low-risk queries from the official examples). Acceptance criteria: The trace should show the sequence of “screenshot observation → tool invocation → new observation”; the coordinate fields should remain within the range of 0–1000, rather than screen pixels.
The model card states clearly: "computer-use" must be used with the official harness; pixel conversion only occurs at the execution boundaries. Mixing coordinate spaces is the number one cause of failure.
Quick List of Pros and Cons: When to Still Use Closed-Source Software / Self-Hosted Loop
| Instructions | When to Use It |
|---|---|
| NorthCUA Harness + Family Weighting | Make computer usage open-source and auditable, and accept the Coordinate Contract. |
| Closed-source computer-use API | We want to outsource SLAs, reduce operational overhead, and ensure compliance with out-of-domain access. |
| Self-Generated Click Loop | There is already a well-established internal protocol; however, it is still recommended to test the coordinate conversion individually. |
Advantages:Open-source Harness, model-agnostic, and separation of long-form notes and evaluations.
Disadvantages:Environmental and permission barriers; the Pro version is large in size; once the coordinate contract is broken, the entire system goes haywire.
Installation and Verification of Minimum Commands (Document Verification)
According to the official README (for illustrative purposes only; subject to change):
git clone https://github.com/beipo-ai/NorthCUA.git cd NorthCUA uv sync --locked --python 3.12 --extra local --extra device uv run --no-sync playwright install chromium uv run --no-sync northcua --help # Desktop example (requires platform permissions; start with read-only/low-risk mode) # uv run --no-sync northcua local desktop run --app-config ... --query '...' --max-steps 15
Explicitly specify in YAML normalized_1000. Web click events follow CSS/DOM references; do not confuse them with desktop coordinate systems.
Key Configuration Points and Troubleshooting (Week 1)
- Dot Deviation/Random Dots:This is most likely due to mixing coordinate systems. Check whether the YAML and tool parameters are still in the range of 0–1000.
- The model chats on its own but expects to get some points:Missing harness. Revert to the official loop.
- Dependencies/Transformers Alert:Follow the risk instructions and pinning procedures in the README; do not silently ignore them.
- Batch cannot be restored:Did not use the official batch/recover functionality. Read the documentation first, then scale up concurrency.
For the first week, only the following will be retained: one read-only desktop smoke test, one web smoke test, and unit tests for the coordinate contract. Tasks involving write permissions will be started after the sandbox is set up.
Explanation of the Evidence:The installation commands and coordinate conventions are excerpted from the public repository/model card (see the meta section for the date of extraction). The author has not completed end-to-end desktop smoke testing on this machine; please verify the functionality yourself in an isolated environment before granting write permissions. If you complete the smoke test, we recommend summarizing the OS, commands, and screenshot of the execution path and submitting them as an official test record.


Comment List (0):
Load More Comments Loading...