# Axilio: drive a real phone from the CLI, then hand back SDK code

Axilio gives you real mobile phones on demand. As the agent, you **explore and drive
a phone live through the `axilio` CLI**, then write the equivalent **SDK script** as
the durable deliverable the user keeps and runs later without you.

## Setup

- The `axilio` CLI must be installed and signed in — `axilio doctor` should be
  all-green. Install with `brew install axilioai/tap/axilio` or
  `curl -fsSL https://axilio.ai/install.sh | sh`; sign in with `axilio login`
  (browser) or by setting `AXILIO_API_KEY`.
- Any command exiting with code 3 means the CLI is not signed in (or the
  credential died). Browser sign-in needs a human: **stop and ask them to run
  `axilio login`**, then continue where you left off. Do not retry the failing
  command in a loop and do not attempt the browser flow yourself. In headless
  or CI setups, `AXILIO_API_KEY` is the human-free alternative.

## Loop: explore and drive via the CLI

Pass `-o json` whenever you want a machine-readable result. In JSON mode, every
successful runnable application command emits exactly one JSON document, including
action commands; warnings and errors remain on stderr. Work one step at a time and
`observe` after actions to confirm the screen changed. Destructive commands do not
prompt in JSON, quiet, or redirected execution, so pass `--yes` deliberately.

```bash
axilio sessions start --phone-type android    # lease a phone (becomes the current session)

axilio phone observe -o json                  # text + UI elements + coordinates on screen
axilio phone find "the search box" -o json    # locate a target by natural-language query
axilio phone tap --query "the search box"     # tap it
axilio phone type "androiddev"                # type into the focused field
axilio phone key enter                        # press enter (the only named key)
axilio phone find-text "Results" -o json      # locate visible text
axilio phone find-all-text "Add to cart"      # every text match on screen (or --pattern <go-regexp>)
axilio phone wait-for "Results" --timeout 15s # wait for text to appear
axilio phone screenshot --out screen.png      # capture the screen
axilio phone send ./photo.jpg --wait          # upload + push media to this phone
axilio files list -o json                     # inspect the library (uploads + captures) and quota

axilio sessions stop <session-id> --yes       # release the phone non-interactively
```

Full verb list and flags: `axilio phone --help`. To drive several phones at once,
`eval "$(axilio sessions start --export)"` pins a phone to the current shell via
`AXILIO_SESSION`, so each terminal drives its own. Phone commands select a session
in this order: `--session`, `AXILIO_SESSION`, the sole active lease, the saved
current-session pointer, then an ambiguity error.

`phone send` keeps the uploaded file in the organization library. Use
`axilio files list --source upload` to discover it, `files push <id> --phone-id <id>`
to reuse it on another phone without re-uploading, and `files delete <id> --yes`
to free quota. Deleting a file removes it from the library only; copies already
on phones, pushed or captured, are not removed.

## Rule: always use semantic selectors, never raw coordinates

**This is the most important rule here.** Find things by what they *are*, not by
where they happened to be on your screen:

```bash
axilio phone tap --query "the search box"   # ✅ do this
axilio phone tap 540 1200                   # ❌ never this
```

A coordinate is only true for one screen size, one layout, one scroll position, one
font scale, one app version. The script you hand back runs later, unattended, on a
*different phone from the pool* than the one you explored with. Hardcoded coordinates
don't fail loudly when any of that shifts — they silently tap the wrong thing, and the
user finds out from the consequences.

Semantic selectors (`find`, `find-text`) re-locate the target on the live screen every
run, so they survive all of it.

The only acceptable use of raw coordinates is a target with genuinely no semantic
handle — a point on a map, a freehand gesture, a canvas. When you must, leave a comment
saying why the semantic path didn't work. "It was easier" is not a reason.

## Deliverable: ask which language, then write the script

Once you've worked the task out live, write a standalone script with the SDK.

**First, ask the user which SDK to write — don't assume:**

> Which SDK should I write this in — Python or Go?

Then follow the matching section below. What you explored maps onto the driver either
way; the language differs in how it installs, handles errors, and releases the phone.

<!-- lang:python -->
### Python

`pip install axilio`

`client.session(...)` is a context manager: it allocates a phone, opens the control
channel, yields a `MobileDriver`, and releases the phone when the `with` block exits —
including on exception. Always drive inside the `with` block.

```python
from axilio.platform import Client
from axilio.drivers import mobile

client = Client()  # reads AXILIO_API_KEY from the environment

with client.session("android") as driver:
    try:
        driver.locator(query="the search box").fill("androiddev")
        driver.key_press("enter")
        driver.get_by_text("Results").wait_for(timeout=15)
        screen = driver.screenshot()  # -> bytes (PNG)
    except mobile.ActionTimeoutError as e:
        # The target never showed up: a dialog, a slow load, a changed app.
        print(f"could not reach the target: {e}")
        raise
```

**Locators.** Find things with a *locator*: a lazy description of a target. Building
one sends nothing; each action or query resolves it against whatever is on screen at
that moment, waits on the phone until it is actionable, then acts, all in one round
trip. There is no separate find step and no stale element to act on.

```python
search = driver.get_by_text("Search")           # visible text, read by OCR
search.fill("androiddev")                        # tap it, then type
driver.locator(query="the blue Continue button").tap()  # natural language, vision model
driver.get_by_text("Loading").wait_for(state="hidden")  # wait for it to disappear
count = driver.get_by_text("Add to cart").count()        # how many right now; never waits
```

- **Prefer `get_by_text(...)`** (fast, deterministic OCR) and fall back to
  `locator(query=...)` for anything text can't pin down (icons, "the first post").
- Selectors by accessibility role or developer id arrive with accessibility support in
  a later release; today every locator is resolved from the screen.
- Every action waits on the phone for the target: 5 s by default, `timeout=` in
  seconds to change it (at most 60). A target that never becomes actionable raises
  `mobile.ActionTimeoutError`. Use `wait_for()` instead of `sleep`.
- A plain `get_by_text(...)` is read by OCR. Anything more (a `query`, `.within(...)`,
  `.has(...)`, `.nth(n)`) is resolved by one vision-model call that is given the whole
  locator, so `nth` and `within` work on natural-language locators too.
- How to resolve belongs to the locator, not the action: pass `model=` and
  `ocr_engine=` when building it (`driver.locator(query=..., model=...)`),
  and give actions only `timeout=`. Refinements keep the locator's options. A locator
  passed into `within()` / `has()` contributes its selector only, so setting options on
  it raises `ValueError`: set them on the outer locator.
- `count()` is the one call that never waits, and it needs a plain text locator (a
  `query`, `within` or `has` raises `mobile.InvalidArgsError`).

**Errors.** Every driver failure is an `AxilioError` subclass. Import the module
(`from axilio.drivers import mobile`) rather than the names directly — `mobile.TimeoutError`
and `mobile.ConnectionError` would otherwise shadow the builtins.

| Exception | When |
|---|---|
| `mobile.ActionTimeoutError` | a locator's wait ran out before the target was actionable (or gone) |
| `mobile.InvalidArgsError` | the call itself is malformed (an unknown key name, for example) |
| `mobile.TimeoutError` | the call's own deadline passed |
| `mobile.DeviceOfflineError` | the phone dropped off the control channel |
| `mobile.NotConnectedError` | driving after the session closed |
| `mobile.AxilioError` | base class — catch this to catch everything |

| Method | Returns | Notes |
|--------|---------|-------|
| `driver.get_by_text(text, exact=False)` | `Locator` | visible text (OCR); the default choice |
| `driver.locator(query=...)` | `Locator` | natural-language description, read by the vision model |
| `driver.press(key)` | `LocatorResult` | press a key against whatever has focus |
| `driver.observe(ocr_engine=None)` | `Screen` | one capture: `screen.texts`, `screen.icons`, `screen.find_text(...)` |
| `driver.type_text(text)` | `None` | type into the focused field |
| `driver.key_press(key)` | `None` | `"enter"` — the only named key today (see below) |
| `driver.tap(coords)` / `driver.long_press(coords)` / `driver.swipe(start, end)` | `None` | coordinate input — see the semantic-selector rule |
| `driver.screenshot()` | `bytes` | PNG |

A `Locator` (call it `loc`) refines and acts. Refinements return a new locator:
`loc.nth(n)`, `loc.first()`, `loc.within(other)`, `loc.has(other)`,
`loc.filter(query=...)`. Actions and queries take only keyword `timeout`:

| Method | Returns | Notes |
|--------|---------|-------|
| `loc.tap()` | `LocatorResult` | wait until actionable, tap its center |
| `loc.fill(text)` | `LocatorResult` | focus the target, then type; prefer this over `tap()` + `type_text()` |
| `loc.press(key)` | `LocatorResult` | focus the target, then press a named key |
| `loc.wait_for(state="visible")` | `LocatorResult \| None` | `state="hidden"` waits until it's gone (returns `None`) |
| `loc.bounding_box()` | `LocatorResult` | where it is: `.bounds` |
| `loc.text()` | `str` | its text |
| `loc.count()` | `int` | matches on the current screen, zero included; never waits |

`observe()` returns a plain snapshot: `screen.find_text(...)` and
`screen.find_all_text(...)` filter that one capture and return data, not something
to act on. To act, use a locator.

**Keys: `"enter"` is the only one.** The named-key table on the device is deliberately
tiny, and anything else (`"home"`, `"back"`, volume, media) raises `mobile.InvalidArgsError`
at run time. Don't guess a key name. To go back or home, drive the on-screen UI with
a locator like a user would.
<!-- /lang:python -->

<!-- lang:go -->
### Go

`go get github.com/axilioai/platform-go`

Go has no context manager; `defer` is the equivalent. Allocate the phone through the
REST client, dial the returned `ControlURL`, and `defer` both the driver close and the
deallocate so the phone is released on every path out — including a panic.

```go
package main

import (
	"context"
	"log"
	"os"
	"time"

	platformgo "github.com/axilioai/platform-go"
	"github.com/axilioai/platform-go/client"
	"github.com/axilioai/platform-go/drivers/mobile"
	"github.com/axilioai/platform-go/option"
)

func main() {
	ctx := context.Background()
	cl := client.NewClient(option.WithAPIKey(os.Getenv("AXILIO_API_KEY")))

	a, err := cl.Phones.Allocate(ctx, &platformgo.PhoneAllocateRequest{
		PhoneType: platformgo.PhoneAllocateRequestPhoneTypeAndroid,
	})
	if err != nil {
		log.Fatalf("allocate: %v", err)
	}
	// Release the phone on every path out.
	defer func() {
		req := &platformgo.PhonesDeallocateRequest{}
		req.SetPhoneID(a.PhoneID)
		if _, derr := cl.Phones.Deallocate(context.Background(), req); derr != nil {
			log.Printf("deallocate: %v", derr)
		}
	}()

	if a.ControlURL == nil {
		log.Fatal("allocation returned no control URL")
	}
	d := mobile.ConnectRemote(*a.ControlURL)
	defer d.Close()

	if _, err := d.Locator(mobile.Query("the search box")).Fill("androiddev"); err != nil {
		if mobile.IsActionTimeout(err) {
			log.Fatal("no search box on screen")
		}
		log.Fatalf("fill: %v", err)
	}
	if err := d.KeyPress(mobile.KeyEnter); err != nil {
		log.Fatalf("key: %v", err)
	}
	results := d.GetByText("Results")
	if _, err := results.WaitFor(mobile.StateVisible, mobile.WithTimeout(15*time.Second)); err != nil {
		log.Fatalf("wait: %v", err)
	}
	png, err := d.Screenshot()
	if err != nil {
		log.Fatalf("screenshot: %v", err)
	}
	_ = png
}
```

**Locators.** Find things with a `*mobile.Locator`: a lazy description of a target.
Building one sends nothing; each action or query resolves it against whatever is on
screen at that moment, waits on the phone until it is actionable, then acts, all in
one round trip. There is no separate find step and no stale element to act on.

- **Prefer `GetByText(...)`** (fast, deterministic OCR) and fall back to
  `Locator(mobile.Query(...))` for anything text can't pin down (icons, "the first post").
- Selectors by accessibility role or developer id arrive with accessibility support in
  a later release; today every locator is resolved from the screen.
- Every action waits on the phone for the target: 5 s by default,
  `mobile.WithTimeout(d)` to change it (at most 60 s). A target that never becomes
  actionable fails with `mobile.IsActionTimeout(err)`. Use `WaitFor` instead of
  `time.Sleep`.
- A plain `GetByText(...)` is read by OCR. Anything more (a `Query`, `Within`, `Has`,
  `Nth`) is resolved by one vision-model call that is given the whole locator.
- How to resolve belongs to the locator, not the action: pass `mobile.Model(...)` and
  `mobile.OCREngine(...)` when building it
  (`driver.Locator(mobile.Query(q), mobile.Model(m))`); actions take only
  `mobile.WithTimeout` (passing any other option to an action does not compile).
  Refinements keep the locator's options. A locator passed into `Within` / `Has`
  contributes its selector only; setting options on it makes the action fail with a
  `mobile.CodeInvalidArgs` error: set them on the outer locator.
- `Count()` is the one call that never waits, and it needs a plain text locator (a
  `Query`, `Within` or `Has` fails with a `mobile.CodeInvalidArgs` error).

**Errors.** Every driver failure is a `*mobile.Error` carrying a `Code` and a
`Retryable` flag. Classify with the helpers rather than matching on strings:

| Helper | When |
|---|---|
| `mobile.IsActionTimeout(err)` | a locator's wait ran out before the target was actionable (or gone) |
| `mobile.IsTimeout(err)` | the call's own deadline passed |
| `mobile.IsDeviceOffline(err)` | the phone dropped off the control channel |
| `mobile.IsRetryable(err)` | the failure is transient — retrying is reasonable |

For anything finer, `errors.As(err, &mobileErr)` and switch on `mobileErr.Code`.

| Method | Returns | Notes |
|--------|---------|-------|
| `driver.GetByText(text, opts...)` | `*Locator` | visible text (OCR); the default choice. `mobile.Exact()` for a whole-string match |
| `driver.Locator(mobile.Query(q))` | `*Locator` | natural-language description, read by the vision model |
| `driver.Press(key, opts...)` | `(LocatorResult, error)` | press a key against whatever has focus |
| `driver.Observe(opts...)` | `(*Screen, error)` | one capture: `Texts`, `Icons`, `screen.FindText(...)` |
| `driver.TypeText(text)` | `error` | type into the focused field |
| `driver.KeyPress(key)` | `error` | `mobile.KeyEnter` — the only named key today (see below) |
| `driver.Tap(c)` / `driver.LongPress(c, ms)` / `driver.Swipe(start, end, ms)` | `error` | coordinate input — see the semantic-selector rule |
| `driver.Screenshot()` | `([]byte, error)` | PNG |

A `*Locator` (call it `loc`) refines and acts. Refinements return a new locator:
`loc.Nth(n)`, `loc.First()`, `loc.Within(other)`, `loc.Has(other)`,
`loc.Filter(query)`. Actions and queries take only `mobile.WithTimeout`:

| Method | Returns | Notes |
|--------|---------|-------|
| `loc.Tap(opts...)` | `(LocatorResult, error)` | wait until actionable, tap its center |
| `loc.Fill(text, opts...)` | `(LocatorResult, error)` | focus the target, then type; prefer this over `Tap` + `TypeText` |
| `loc.Press(key, opts...)` | `(LocatorResult, error)` | focus the target, then press a named key |
| `loc.WaitFor(state, opts...)` | `(*LocatorResult, error)` | `mobile.StateVisible`, or `mobile.StateHidden` to wait until it's gone (result is nil) |
| `loc.BoundingBox(opts...)` | `(LocatorResult, error)` | where it is: `.Bounds` |
| `loc.Text(opts...)` | `(string, error)` | its text |
| `loc.Count(opts...)` | `(int, error)` | matches on the current screen, zero included; never waits |

`Observe()` returns a plain snapshot: `screen.FindText(...)` and
`screen.FindAllText(...)` filter that one capture and return data, not something to
act on. To act, use a locator.

**Keys: `mobile.KeyEnter` is the only one.** The named-key table on the device is
deliberately tiny, and anything else (`"home"`, `"back"`, volume, media) is rejected at
run time. Don't guess a key name; the package exports exactly the keys that work. To go
back or home, drive the on-screen UI with a locator like a user would.
<!-- /lang:go -->

## Store the script as a workflow and run it

The script doesn't have to live only in the user's repo — Axilio can hold it as a
*workflow*: versioned, runnable on demand, no agent in the loop.

```bash
axilio workflows create my-flow --platform android --code flow.py  # revision 1 from your file
axilio workflows pull <id> --out flow.py                     # fetch the current code
axilio workflows push <id> flow.py -m "handle cookie banner" # save a new revision
axilio workflows revisions <id>                              # history; workflows restore <id> <rev-id> brings one back
axilio runs start <workflow-id> --watch                      # run it and stream output until it finishes
axilio runs watch <run-id>                                   # attach to a running run, or replay a finished one
```

`--watch` streams the run's logs and completed steps live (push-delivered while
the session is active, durable archive as automatic fallback) and exits with
the run's outcome: 0 completed, 1 failed (with the error message), 7 cancelled.
With `-o json` the stream is newline-delimited JSON — one object per telemetry
frame, then a final `{"watch_end": true, ...}` summary; consume it line by line.
Pushing unchanged code is a reported no-op; restoring an old revision creates a
new one, so nothing is ever lost.

## Inspect what happened: sessions, traces, captures

```bash
axilio sessions get <id> -o json        # status, phone, duration, tags, recording URL
axilio sessions trace <id> -o json      # ordered spans/logs with durations and billed costs
axilio sessions trace <id> --follow     # archived trace, then live frames until the session ends
axilio sessions list --history --status failed --started-after 2026-08-01
axilio sessions files <session-id>      # files the session captured off the phone
axilio files download <id> --out file   # save a ready file (upload or capture) locally
```

A trace is the same data the dashboard's session viewer shows — every SDK call
and inference as spans, costs joined per call. `--follow` keeps tailing live
frames after the archive; live rows show COST as "-" (billing joins at read
time), and with `-o json` the output becomes newline-delimited JSON ending in a
`{"trace_end": true, ...}` summary carrying the final cost maps. Captures are files
that landed on the phone during a session; they sit in the same library as
uploads with `source=capture`. List them org-wide with
`axilio files list --source capture` (filter by `--session`, `--mime`, size/date
bounds), and free space with `files delete <id> --yes`. A skipped or failed
capture is a visible row with a reason, not an absence.

## Money and capacity

```bash
axilio billing balance                  # remaining credit + plan — check before allocating
axilio usage metrics --from 2026-08-01  # org spend summary (compute minutes, spend by product)
axilio usage inferences --from 2026-08-01 --session <id>  # per-call vision costs and latency
axilio runs stats <workflow-id>         # run count + success rate
```

Allocation fails on an empty balance; checking first beats debugging it after.
The CLI has no billing writes — adding funds and plan changes stay in the
dashboard. Time flags accept RFC 3339 or bare `YYYY-MM-DD` (midnight UTC).

## Dedicated phones

```bash
axilio phones rename <phone-id> "inventory-phone-2"  # nickname shown in listings
axilio phones preview <phone-id> --out preview.jpg   # current screen, no session needed
axilio phones wipe <phone-id> --yes                  # factory-reset an idle dedicated phone
```

## Rules

- **Semantic selectors, never raw coordinates.** See the rule above — it's the
  difference between a script that keeps working and one that silently misfires.
- Explore with the CLI; the **SDK script is the deliverable** — the user runs it
  later with no agent in the loop, so it must handle its own failures.
- Ask which language before writing. Don't assume Python.
- One action per step, then `observe` to verify.
- Always release the phone: `axilio sessions stop <id>` when exploring; in the
  script, the Python `with` block or the Go `defer` does it for you.