Agenteract is an experimental bridge that lets coding agents view and interact with running applications - Including React Native / Expo, React, Kotlin Multi Platform, and Swift UI. It's possible to add support for your favorite stack by replicating the existing framework implementations.

Agenteract exposes the app's internal state, view hierarchy, and actionable controls over a secure WebSocket, enabling agents (or test harnesses) to observe and trigger UI events just like a developer or user would.

Agent Based

The easiest way to get started is to let your coding agent do the work. First, create an AGENTS.md file:

npx @agenteract/agents md

You can also specify the name:

npx @agenteract/agents md CLAUDE.md

This appends the Agenteract instructions to your agents markdown.

Next, restart your coding CLI or open a new chat tab (Cursor), then ask it to complete the setup:

"Please add Agenteract support and make sure it works."

Manual

First, you'll need to install the Agenteract CLI. This tool manages the communication between the AI agent and your local development servers.

npm install -g @agenteract/cli

Next, install the appropriate package for your project.

For React Native (Expo):

npm install @react-native-async-storage/async-storage expo-linking

For React (Vite):

npm install @agenteract/react

React Native, Expo, React all use the same import @agenteract/react. Previously Expo and Vite specific packages were used to provide dev server wrappers, but this is now generalized.

For Flutter:

Add to your pubspec.yaml:

dependencies:
  agenteract:
    git:
      url: https://github.com/agenteract/agenteract.git
      path: packages/flutter

Next, install AGENTS.md - This will allow your coding assistant to understand how Agenteract works.

If you already have an AGENTS.md, our contents will be appended.

You can also specify the output filename.

npx @agenteract/agents [FILENAME.md]

Now you can reference the file for your agent in a message, or restart the CLI for it to take effect.

At this point you can ask your agent to start setting up Agenteract.

The command below will create an initial agenteract.config.js, or add entries to an existing configuration.

New Format (Generic Dev Server - Recommended):

npx @agenteract/cli add-config <path> <projectName> <command> [port] --scheme myapp

port is auto assigned if not provided. This is the port that Agenteract uses to communicate internally, it's not where the dev server hosts files.

--scheme myapp: Scheme used for QR code / deep link pairing.

Parameters:

Examples:

# Expo Go - Supports exp:// scheme by default, the CLI will run from CWD
npx @agenteract/cli add-config . expo-app "npx expo" --scheme exp

# The following examples specify relative paths as would be the case in a mono repo
# Next.js app with explicit port
npx @agenteract/cli add-config ./apps/web next-app "npm run dev" --port 8791

# Remix app with auto-assigned port
npx @agenteract/cli add-config ./apps/remix remix-app "remix dev"

# Custom dev server
npx @agenteract/cli add-config ./apps/custom my-app "pnpm start:dev"

Note: The new generic format supports any dev server command, making Agenteract framework-agnostic.

Here is an example configuration for a monorepo containing multiple projects:

// agenteract.config.js
export default {
  /**
   * The port for the central Agenteract server.
   * The agent connects to this port.
   */
  port: 8766,

  /**
   * An array of projects to manage.
   */
  projects: [
    {
      // A unique identifier for this app. Used for targeting commands.
      name: 'expo-app',
      // The path to the app's root directory, relative to this config file.
      path: './examples/expo-example',
      // Generic dev server configuration
      devServer: {
        command: 'npx expo start',
        port: 8790,
      },
      "scheme": "myapp"
    },
    {
      name: 'react-app',
      path: './examples/react-example',
      devServer: {
        command: 'npx vite',
        port: 8791,
      }
    },
    {
      name: 'flutter-app',
      path: './examples/flutter_example',
      devServer: {
        command: 'flutter run',
        port: 8792,
        scheme: 'my-flutter-app',
        validation: {
          fileExists: ['pubspec.yaml'],
          commandInPath: 'flutter',
        }
      }
    },
    {
      name: 'next-app',
      path: './apps/web',
      devServer: {
        command: 'npm run dev',
        port: 8793,
      }
    },
    {
      name: 'swift-app',
      path: './examples/swift-app',
      type: 'native'  // Native apps don't have dev servers
    }
  ],
};

To allow Agenteract to 'see' and interact with your application, you need to add the AgentDebugBridge component to your app's entry point.

Platform-Specific Setup

For physical device testing (React Native, Expo, Swift, Kotlin), you'll also need to configure deep linking to enable secure pairing. See platform-specific instructions below.

For React Native / Expo - App.tsx:

React Native, Expo, React all use the same import @agenteract/react.

Previously Expo and Vite specific packages were used to provide dev server wrappers, but this is now generalized.

import { View, Text } from 'react-native';
import { AgentDebugBridge } from '@agenteract/react';

export default function App() {
  return (
    <View style={{ flex: 1 }}>
      {/* Your existing application */}
      <Text>Welcome to my app!</Text>

      {/* Add the AgentDebugBridge */}
      { __DEV__ && <AgentDebugBridge projectName="expo-app" /> }
    </View>
  );
}

For React (Vite) - src/App.tsx:

import { AgentDebugBridge } from '@agenteract/react';

function App() {
  return (
    <>
      {/* Your existing application */}
      <h1>Welcome to my app!</h1>

      {/* Add the AgentBridge */}
      { __DEV__ && <AgentDebugBridge projectName="vite-app" /> }
    </>
  );
}

export default App;

For Flutter lib/main.dart:

Packages:

agenteract (Git or local path - not yet on pub.dev)

Installation:

dependencies:
  agenteract:
    git:
      url: https://github.com/agenteract/agenteract.git
      path: packages/flutter

The following can be installed either in the app, or at the monorepo root if applicable:

@agenteract/cli, @agenteract/server, @agenteract/agents

Usage:

import 'package:agenteract/agenteract.dart';
import 'package:flutter/foundation.dart';
// ...
if (kDebugMode) {
  return AgentDebugBridge(
    projectName: 'myFlutterApp',
    child: MyApp(),
  );
}

Making widgets interactive:

// Use the .withAgent() extension on any widget
ElevatedButton(
  onPressed: () => print('clicked'),
  child: Text('Click me'),
).withAgent('submit-button', onTap: () => print('clicked'))

// Text input
TextField(
  onChanged: (text) => print(text),
).withAgent('username-input', onChangeText: (text) => print(text))

For Swift UI

See agenteract-swift

For Kotlin Multiplatform (Compose Multiplatform)

Consult packages/kotlin/README.md for installation and usage instructions.

Usage:

import io.agenteract.AgentDebugBridge
// ...
AgentDebugBridge(projectName = "kmp-app")

Making composables interactive:

import io.agenteract.agent

// Button with tap handler
Button(
    onClick = { handleClick() },
    modifier = Modifier.agent(
        testID = "submit-button",
        onTap = { handleClick() }
    )
) {
    Text("Submit")
}

// Text input
var text by remember { mutableStateOf("") }
TextField(
    value = text,
    onValueChange = { text = it },
    modifier = Modifier.agent(
        testID = "username-input",
        onChangeText = { text = it }
    )
)

With your configuration in place and your app instrumented, you can now start Agenteract.

Open a terminal and run the following command from the root of your project (where your agenteract.config.js is located):

npx @agenteract/cli dev

This command will:

Connecting Devices

For Simulators/Emulators and Web Apps:

Applications automatically connect to localhost:8765 - no additional setup needed!

For Physical Devices (React Native, Expo, Swift, Kotlin):

Physical devices require deep link pairing for secure connections:

1. Configure your app's URL scheme (if not already done):

npx @agenteract/cli add-config . my-app native --scheme myapp

# For Expo Go, use scheme exp:
npx @agenteract/cli add-config . my-app expo --scheme exp

2. Start the dev server (if not already running):

npx @agenteract/cli dev

3. Connect your physical device:

npx @agenteract/cli connect

4. Scan the QR code displayed in the terminal with your device camera

The deep link will configure your app with the server's IP address, port, and authentication token. This configuration is saved permanently and used for all future connections.

Device information is stored in .agenteract-runtime.json. This file should not be checked in to SCM.

Platform-Specific Deep Linking Setup:

Agent Interaction

AI agents can now connect to the Agenteract server using the tools described in AGENTS.md. The agent can view your app's component hierarchy and perform actions like tapping buttons or typing into text fields.

agenteract.config.js should not be tracked as it may be specific to your local environment.

Your agent is now ready to Agenteract!

This guide covers how to set up the local development environment to run the Expo example app and the agent server.

First, clone the repository and ensure you have pnpm installed.

git clone https://github.com/agenteract/agenteract.git
cd agenteract
git submodule update --init --recursive
npm install -g pnpm

Install all dependencies for the monorepo and build the packages from the root directory.

npm install -g pnpm
pnpm install
pnpm build

We need to use linking to work locally:

Usage:

Usage:

cd packages/server
pnpm link --global
cd packages/agents
pnpm link --global
cd packages/cli
pnpm link --global
cd packages/pty
pnpm link --global

Start the Agenteract development environment using the unified CLI:

pnpm agenteract dev

# once published, you can use:
npx @agenteract/cli dev

This will:

The multiplexed output will show logs from all your running applications in a single terminal.

Once the app is running, it will automatically connect to the agent server. You should see connection logs in the multiplexed output.

You can manually simulate an agent command to test the connection:

curl -s -X POST http://localhost:8766/expo-app -d '{"action":"getViewHierarchy"}'

The server will forward this to the app, which responds with a JSON payload of its view hierarchy.

This step creates or appends to your AGENTS.md file. This informs coding agents how to interact with the app.

npx @agenteract/agents md [dest] # You can specific the name, eg GEMINI.md

If you are using a separate agent to your IDE, start it now, otherwise you can use the built in agent (Tested with Cursor, Gemini CLI)

Issue some instructions. You might need to prime the agent the first time

You can use the Get View Hieararchy tool to inspect the current app state.
Add a button that disappears when it is clicked.
Confirm that it works using a simulated action.

Agents should view the current hierarchy, modify the code, view again, simulate a tap, then confirm that the button disappeared by viewing the hierarchy one final time.

Because packages/agents/AGENTS.md contains instructions about how to interact with the app, you don't need to explicitly tell it to use the AgentDebugBridge.

This project uses Jest for testing.

To run the tests for all packages, use the following command from the root directory:

pnpm test

To run the tests for a specific package, use the --filter flag:

pnpm --filter @agenteract/react test

Tests are run automatically on every push and pull request to the main branch using GitHub Actions.

Integration tests verify that packages can be installed and used correctly after publication. They use Verdaccio , a lightweight private npm registry running in Docker.

Local testing workflow:

# Start local npm registry
pnpm verdaccio:start

# Build and publish packages
pnpm verdaccio:publish

# Run integration tests
pnpm test:integration

# Clean up
pnpm verdaccio:stop

GitHub Actions: Integration tests run automatically on PRs and pushes to main and release/** branches using Verdaccio as a service container.

Authentication: Uses expect to automate the authentication process. See docs/VERDACCIO_AUTH.md for details.

See docs/INTEGRATION_TESTING.md for complete information.