AgentEnv

AgentEnv is an open-source framework from engineers at Scale AI for building RL environments and running and grading agents in them. The froog sandbox provider runs each environment and each agent in its own froog sandbox: an Ubuntu VM with Docker.

Status

AgentEnv ships a provider for boat; the froog provider is written in the same shape (config, sizes, labels, idempotent creates, cleanup, reconnect, uploads) and is on its way upstream. Until it's merged, install it as a plugin: AgentEnv loads out-of-tree providers from the agent_env.sandbox_providers entry point, or from impl = "module:Class" in your config.

Set it up

$ pip install agentenv-framework httpx
$ export FROOG_API_KEY=froog_...     # or: froog api-key create agentenv --preset ci

# .agentenv/config.toml
[sandbox.providers.froog]
impl = "agent_env.providers.sandbox_providers.froog.provider:FroogSandboxProvider"   # until it's built in

[sandbox.providers.froog.config]
api_key = "env:FROOG_API_KEY"      # env: or secret:, never the key itself
# api_url = "https://api.froog.tech"

[sandbox]
default = "froog"        # environments and deploy_sandbox steps
agent_default = "froog"  # agents

$ agent-env run hello --sandbox froog      # the built-in hello task, no model needed

For one run only: agent-env run --sandbox froog, or agent-env task run --env-sandbox froog --agent-sandbox froog. "froog,local" falls back to the next provider when a create fails.

What happens on froog

AgentEnv step	On froog
Create an env or agent	A sandbox of the smallest size that covers the CPU, memory and disk asked for (`small` 2 vCPU / 4 GiB / 51 GB, `default` 4 / 8 / 51, `large` 8 / 16 / 111), then waits for Docker and Compose.
Exposed ports	Each gets a public HTTPS URL (`*.froogusercontent.com`).
Run ends	AgentEnv deletes the sandbox.
Run crashes or is cancelled	The provider finds the sandbox by its create label and deletes it.
Nothing deletes it	froog stops it at AgentEnv's timeout (2 hours by default) and deletes it a minute later.

Each sandbox starts without your account's environment (no variables, secret files, repos or GitHub token) and without snapshots. Every sandbox carries agentenv.managed=true, and AgentEnv attribution becomes agentenv.attr.* labels, so froog list --label agentenv.attr.run_id=… finds a run's sandboxes and froog usage NAME prices them. When froog refuses a create at a start limit, the provider waits (honoring Retry-After) and tries again, up to 6 times.

Not available

An image override: every sandbox boots froog's Ubuntu image.
A network_policy that restricts egress: froog has no egress allowlist yet. The provider refuses rather than create a less isolated sandbox than asked.
Commands run up to an hour each (froog's exec limit); longer ones come back with exit code 124.

Start free trial Read the quickstart

7 days free: 25 hours of running time, 2 sandboxes at once. Then from $0.0137 an hour, billed per second.