Mobile Agent

Build, run, preview, and fix Expo, React Native, and Flutter apps from the Navin Dev workbench. The agent detects the stack, doctors the Android/Node toolchain, starts the packager, and can drive a live Android device or emulator. Screen capture and input injection prefer the Rust navin_core.MobilePreviewSession accelerator, with a pure-Python fallback when the native extension is not built.

Why You Need It

Web-oriented Dev mode already edits files and runs shells. Mobile work adds:

stack detection (Expo vs bare React Native vs Flutter);
environment checks (Node, adb, Android SDK, AVD);
one-click / one-command run of Metro, Expo, or flutter run;
an integrated Mobile preview tab (screen + logcat + light metrics);
tap / swipe / key injection so the agent can verify UI changes.

iOS still requires a Mac with Xcode. Visual regression screenshots and deep source mapping (React DevTools / Flutter Inspector) are not in this release.

Quick start

Open the Dev module (#/code) on an Expo, React Native, or Flutter project.
Run /mobile android in chat (or click Run Mobile in the workbench toolbar).
Wait until adb devices shows a device or emulator.
Ask the agent to start preview (mobile(action="preview_start")) - the Mobile tab opens and starts automatically. You can still open the tab and click Start preview yourself.
Click the image to tap; drag to swipe.

Rebuild the native accelerator after pulling these changes:

make native

Without navin_core, preview still works via the Python adb path (usually slower).

Slash command

Command	Args	Effect
`/mobile`	`[android\|ios\|web\|metro\|doctor]`	Loads `mobile-dev`, runs detect → doctor → run (or doctor-only). Tracked on the board.

Examples:

/mobile
/mobile android
/mobile doctor
/mobile web

`mobile` tool

Prefer this tool over inventing adb/Metro commands.

Packager lifecycle

Action	Purpose
`detect`	Identify Expo / React Native / Flutter and package manager
`doctor`	Check Node, npm/yarn/pnpm/bun, npx, Java, adb, SDK, emulator, devices
`setup`	Discover adb/SDK (Windows `%LOCALAPPDATA%\\Android\\Sdk` + user platform-tools download, macOS Homebrew + `~/Library/Android/sdk`, Linux, WSL). Auto-install when possible; otherwise return OS-specific steps
`devices`	List `adb devices` and AVDs
`plan`	Show commands without starting anything
`run`	Start packager (and optionally an AVD)
`logs`	Poll packager session output
`stop`	Terminate packager / emulator / agent preview session

mobile(action="detect")
mobile(action="doctor")
mobile(action="run", target="android")
mobile(action="logs")
mobile(action="stop")

target for run / plan: android (default), ios, web, metro.

Interactive Android preview

Action	Purpose
`preview_start`	Open an agent-side preview session (Rust if available)
`preview_stop`	Stop that session
`screenshot`	Grab the latest PNG frame summary
`tap`	Tap device pixels (`x`, `y`)
`swipe`	Swipe (`x`, `y`, `x2`, `y2`, optional `duration_ms`)
`key`	Android keyevent (`BACK`, `HOME`, `ENTER`, …)
`text`	Type text into the focused field
`ui_dump`	UIAutomator XML hierarchy (testID / text / bounds)
`metrics`	FPS estimate, memory, CPU, resolution

mobile(action="preview_start")
mobile(action="tap", x=540, y=1200)
mobile(action="ui_dump")
mobile(action="metrics")

The WebUI Mobile tab uses the same stack over WebSocket (mobile_preview_* envelopes), independent of the agent tool session.

Skill

mobile-dev (navin/skills/mobile-dev/SKILL.md) teaches the operating loop: detect → doctor → run → edit → logs / preview → verify. fullstack-dev points mobile workspaces at this skill.

Architecture

WebUI Mobile tab  ──WS mobile_preview_*──►  gateway (Python)
                                              │
                         open_preview() ──────┤
                                              ▼
                    navin_core.MobilePreviewSession (Rust)
                         or PythonPreviewSession (fallback)
                                              │
                                              ▼
                                         adb screencap / logcat / input

Piece	Location
Detection / doctor / run plan	`navin/mobile/{detect,doctor,run}.py`
Preview session API	`navin/mobile/preview.py`
Rust accelerator	`navin-core/src/mobile.rs` → `MobilePreviewSession`
Agent tool	`navin/agent/tools/mobile.py`
WebSocket manager	`navin/webui/mobile_preview_ws.py`
Dispatch	`navin/channels/websocket.py`
UI panel	`webui/src/components/dev/DevMobilePreview.tsx`

Native access follows the usual pattern: navin.utils.native.native(), with NAVIN_DISABLE_NATIVE=1 forcing the Python path.

Workbench UI

In the Dev workbench tab strip:

Code · Preview (web iframe) · Mobile · Graph · Board · …

Mobile shows:

live PNG frames from the device;
logcat sidebar;
FPS / memory / CPU readouts;
Back / Home / Recents keys;
click = tap, drag = swipe.

Localhost-only (same rule as integrated terminals).

Stack notes

Expo

Detected via expo dependency and/or app.json / app.config.*. Default run: npx expo start --android (or package-manager equivalent).

React Native CLI

Detected via react-native without Expo. Uses project scripts (start, android) when present.

Flutter

Detected via pubspec.yaml + Flutter SDK. Run uses flutter run -d android (or chrome / ios). Preview uses the same Android adb path once a device is online.

Limitations

iOS: needs macOS + Xcode; Linux/Windows should use Android or Expo web.
Source mapping: ui_dump helps match controls by text/testID/bounds; not a full component inspector.
Performance metrics: approximate (screencap FPS, dumpsys/top samples), not a profiler.
WSL + USB: device bridging often needs usbipd on Windows; emulators are usually simpler.

Related docs

Chat from a phone (usage PWA, not this preview): mobile-usage-app.md
Dev workbench overview: navin_dev
English workbench page: navin_dev/en/workbench.md
French Mobile page: navin_dev/fr/mobile.md
Architecture / native: architecture.md