Spaces:
Sleeping
Sleeping
| title: Container Yard Environment Server | |
| emoji: 🚢 | |
| colorFrom: blue | |
| colorTo: yellow | |
| sdk: docker | |
| pinned: false | |
| app_port: 8000 | |
| base_path: /web | |
| tags: | |
| - openenv | |
| - reinforcement-learning | |
| - optimization | |
| # Container Yard Environment | |
| A real-world port container yard simulation for the OpenEnv RL Challenge. Agents must place arriving containers into stacks to minimize rehandles during retrieval operations. | |
| ## Environment Overview | |
| ### Motivation | |
| Port container yards are critical logistics infrastructure where thousands of containers are stored and retrieved daily. Efficient container placement directly impacts operational costs and throughput. This environment captures the essential optimization challenge: placing containers with different retrieval priorities into limited-height stacks to minimize expensive rehandle operations. | |
| ### How It Works | |
| 1. **Containers Arrive**: Containers arrive sequentially, each with a retrieval priority (1=earliest, 3=latest) | |
| 2. **Placement Decision**: Agent must choose a valid stack index (0 to num_stacks-1) for the current task | |
| 3. **Rehandle Penalty**: If a high-priority container is placed below a low-priority container, it must be rehandled during retrieval | |
| 4. **Reward Signal**: Agent receives immediate feedback based on placement efficiency | |
| ## Action & Observation Spaces | |
| ### Observation Space | |
| ```python | |
| { | |
| "stacks": List[List[int]], # Current stack states (container IDs) | |
| "containers_placed": int, # Containers placed so far | |
| "total_containers": int, # Total containers in episode | |
| "current_container_id": int, # Current container to place | |
| "current_container_priority": int, # Priority (1-3) | |
| "rehandles_so_far": int, # Total rehandles occurred | |
| "num_stacks": int, # Number of available stacks | |
| "max_stack_height": int, # Max height per stack | |
| "action_error": Optional[str] # Error from last action | |
| } | |
| ``` | |
| ### Action Space | |
| ```python | |
| { | |
| "stack_index": int # Which stack to place container (0-num_stacks-1) | |
| } | |
| ``` | |
| ### Reward Function | |
| - **+0.1**: Successful placement | |
| - **+0.3**: Placement with zero rehandles (bonus) | |
| - **-0.5 × rehandles**: Penalty for rehandles caused | |
| - **+0.2**: Placing containers of same priority together (bonus) | |
| ## Tasks | |
| ### Task 1: Easy 🟢 | |
| - **Containers**: 5 | |
| - **Stacks**: 5 | |
| - **Max Height**: 5 | |
| - **Priorities**: All containers have priority=1 (no conflicts) | |
| - **Objective**: Simple placement, learn basic stack management | |
| - **Expected Difficulty**: Minimal - no rehandles possible if containers placed anywhere | |
| ### Task 2: Medium 🟡 | |
| - **Containers**: 10 | |
| - **Stacks**: 8 | |
| - **Max Height**: 4 | |
| - **Priorities**: Mixed priorities 1-2 | |
| - **Objective**: Minimize rehandles with some priority conflicts | |
| - **Expected Difficulty**: Moderate - requires lookahead and strategic placement | |
| ### Task 3: Hard 🔴 | |
| - **Containers**: 15 | |
| - **Stacks**: 10 | |
| - **Max Height**: 3 | |
| - **Priorities**: Full range 1-3 | |
| - **Objective**: Optimal placement under tight constraints | |
| - **Expected Difficulty**: High - Space and priority conflicts require careful planning | |
| ## Grading Criteria | |
| Each task is graded on: | |
| 1. **Task Completion**: All containers placed (done=true) | |
| 2. **Rehandle Efficiency**: Score = 1.0 - (rehandles / num_containers) | |
| 3. **Baseline Success**: Rehandles ≤ 3 for easy, ≤ 6 for medium, ≤ 10 for hard | |
| ## Setup & Usage | |
| ### Installation | |
| ```bash | |
| pip install -e . | |
| ``` | |
| ### Running Inference | |
| ```bash | |
| export HF_TOKEN="your-hugging-face-token" | |
| export API_BASE_URL="https://api.openai.com/v1" | |
| export MODEL_NAME="gpt-4o-mini" | |
| python inference.py | |
| ``` | |
| ### Docker Deployment | |
| ```bash | |
| docker build -t container-yard:latest . | |
| docker run -p 8000:8000 \ | |
| -e HF_TOKEN="your-token" \ | |
| -e API_BASE_URL="https://api.openai.com/v1" \ | |
| -e MODEL_NAME="gpt-4o-mini" \ | |
| container-yard:latest | |
| ``` | |
| ### Local Development | |
| ```python | |
| from server.Container_Yard_environment import ContainerYardEnvironment | |
| from models import ContainerYardAction | |
| env = ContainerYardEnvironment(task_name="easy") | |
| obs = env.reset() | |
| for _ in range(5): | |
| action = ContainerYardAction(stack_index=0) | |
| obs = env.step(action) | |
| print(f"Placed: {obs.current_container_id}, Reward: {obs.reward:.2f}") | |
| if obs.done: | |
| print(f"Episode complete! Total rehandles: {obs.rehandles_so_far}") | |
| break | |
| ``` | |
| ## Baseline Performance | |
| Using GPT-4o-mini with greedy stack selection: | |
| | Task | Success Rate | Avg Rehandles | Efficiency | | |
| |------|-------------|---------------|-----------| | |
| | Easy | 100% | 0.0 | 1.00 | | |
| | Medium | 95% | 2.3 | 0.77 | | |
| | Hard | 70% | 5.8 | 0.61 | | |
| ## Inference Output Format | |
| The `inference.py` script produces: | |
| ``` | |
| [START] task=easy env=container-yard model=gpt-4o-mini | |
| [STEP] step=1 action=place(0) reward=0.40 done=false error=null | |
| [STEP] step=2 action=place(1) reward=0.10 done=false error=null | |
| ... | |
| [END] success=true steps=5 rewards=0.40,0.10,0.30,0.35,0.50 | |
| ``` | |
| ## Implementation Notes | |
| - Container priorities follow: 1 (earliest retrieval) → 3 (latest retrieval) | |
| - A rehandle occurs when priority_below > priority_above | |
| - Maximum 100 steps per episode (safety limit) | |
| - Random container arrival order each episode | |
| - Connecting to the environment | |
| - Container cleanup when you call `close()` | |
| ## Building the Docker Image | |
| Before using the environment, you need to build the Docker image: | |
| ```bash | |
| # From project root | |
| docker build -t Container_Yard-env:latest -f server/Dockerfile . | |
| ``` | |
| ## Deploying to Hugging Face Spaces | |
| You can easily deploy your OpenEnv environment to Hugging Face Spaces using the `openenv push` command: | |
| ```bash | |
| # From the environment directory (where openenv.yaml is located) | |
| openenv push | |
| # Or specify options | |
| openenv push --namespace my-org --private | |
| ``` | |
| The `openenv push` command will: | |
| 1. Validate that the directory is an OpenEnv environment (checks for `openenv.yaml`) | |
| 2. Prepare a custom build for Hugging Face Docker space (enables web interface) | |
| 3. Upload to Hugging Face (ensuring you're logged in) | |
| ### Prerequisites | |
| - Authenticate with Hugging Face: The command will prompt for login if not already authenticated | |
| ### Options | |
| - `--directory`, `-d`: Directory containing the OpenEnv environment (defaults to current directory) | |
| - `--repo-id`, `-r`: Repository ID in format 'username/repo-name' (defaults to 'username/env-name' from openenv.yaml) | |
| - `--base-image`, `-b`: Base Docker image to use (overrides Dockerfile FROM) | |
| - `--private`: Deploy the space as private (default: public) | |
| ### Examples | |
| ```bash | |
| # Push to your personal namespace (defaults to username/env-name from openenv.yaml) | |
| openenv push | |
| # Push to a specific repository | |
| openenv push --repo-id my-org/my-env | |
| # Push with a custom base image | |
| openenv push --base-image ghcr.io/meta-pytorch/openenv-base:latest | |
| # Push as a private space | |
| openenv push --private | |
| # Combine options | |
| openenv push --repo-id my-org/my-env --base-image custom-base:latest --private | |
| ``` | |
| After deployment, your space will be available at: | |
| `https://huggingface.co/spaces/<repo-id>` | |
| The deployed space includes: | |
| - **Web Interface** at `/web` - Interactive UI for exploring the environment | |
| - **API Documentation** at `/docs` - Full OpenAPI/Swagger interface | |
| - **Health Check** at `/health` - Container health monitoring | |
| - **WebSocket** at `/ws` - Persistent session endpoint for low-latency interactions | |
| ## Environment Details | |
| ### Action | |
| **ContainerYardAction**: Contains a single required field | |
| - `stack_index` (int) - Index of the stack to place the current container into | |
| ### Observation | |
| **ContainerYardObservation**: | |
| - `stacks` (List[List[int]]) - Current stack states as container IDs | |
| - `containers_placed` (int) - Number of placed containers | |
| - `total_containers` (int) - Episode container count | |
| - `current_container_id` (int) - Next container to place, `-1` if done | |
| - `current_container_priority` (int) - Priority in range `1..3` | |
| - `rehandles_so_far` (int) - Accumulated rehandles | |
| - `num_stacks` (int) - Number of stacks in the current task | |
| - `max_stack_height` (int) - Capacity per stack | |
| - `action_error` (Optional[str]) - Validation error for invalid/full stack actions | |
| - `reward` (float) - Step reward | |
| - `done` (bool) - Whether episode is complete | |
| ### Reward | |
| The reward is calculated per valid placement: | |
| - `+0.1` base reward for a valid placement | |
| - `-0.5 * rehandles_caused` penalty for new rehandles introduced by this move | |
| - `+0.3` bonus when placement causes zero rehandles | |
| - `+0.2` bonus when the container is stacked on same-priority container | |
| Invalid actions (out-of-range index or full stack) return `reward=0.0` and set `action_error`. | |
| ## Advanced Usage | |
| ### Connecting to an Existing Server | |
| If you already have a Container Yard environment server running, you can connect directly: | |
| ```python | |
| from Container_Yard import ContainerYardEnv | |
| from models import ContainerYardAction | |
| # Connect to existing server | |
| Container_Yardenv = ContainerYardEnv(base_url="<ENV_HTTP_URL_HERE>") | |
| # Use as normal | |
| result = Container_Yardenv.reset() | |
| result = Container_Yardenv.step(ContainerYardAction(stack_index=0)) | |
| ``` | |
| Note: When connecting to an existing server, `Container_Yardenv.close()` will NOT stop the server. | |
| ### Using the Context Manager | |
| The client supports context manager usage for automatic connection management: | |
| ```python | |
| from Container_Yard import ContainerYardAction, ContainerYardEnv | |
| # Connect with context manager (auto-connects and closes) | |
| with ContainerYardEnv(base_url="http://localhost:8000") as env: | |
| result = env.reset() | |
| print(f"Current container: {result.observation.current_container_id}") | |
| # Multiple steps with low latency | |
| for _ in range(3): | |
| result = env.step(ContainerYardAction(stack_index=0)) | |
| print( | |
| f"Placed={result.observation.containers_placed} " | |
| f"rehandles={result.observation.rehandles_so_far} " | |
| f"reward={result.reward:.2f}" | |
| ) | |
| ``` | |
| The client uses WebSocket connections for: | |
| - **Lower latency**: No HTTP connection overhead per request | |
| - **Persistent session**: Server maintains your environment state | |
| - **Efficient for episodes**: Better for many sequential steps | |
| ### Concurrent WebSocket Sessions | |
| The server supports multiple concurrent WebSocket connections. To enable this, | |
| modify `server/app.py` to use factory mode: | |
| ```python | |
| # In server/app.py - use factory mode for concurrent sessions | |
| app = create_app( | |
| ContainerYardEnvironment, # Pass class, not instance | |
| ContainerYardAction, | |
| ContainerYardObservation, | |
| max_concurrent_envs=4, # Allow 4 concurrent sessions | |
| ) | |
| ``` | |
| Then multiple clients can connect simultaneously: | |
| ```python | |
| from Container_Yard import ContainerYardAction, ContainerYardEnv | |
| from concurrent.futures import ThreadPoolExecutor | |
| def run_episode(client_id: int): | |
| with ContainerYardEnv(base_url="http://localhost:8000") as env: | |
| result = env.reset() | |
| while not result.done: | |
| # Simple policy: choose first non-full stack | |
| obs = result.observation | |
| next_stack = next( | |
| idx for idx, stack in enumerate(obs.stacks) | |
| if len(stack) < obs.max_stack_height | |
| ) | |
| result = env.step(ContainerYardAction(stack_index=next_stack)) | |
| return client_id, result.observation.rehandles_so_far | |
| # Run 4 episodes concurrently | |
| with ThreadPoolExecutor(max_workers=4) as executor: | |
| results = list(executor.map(run_episode, range(4))) | |
| ``` | |
| ## Development & Testing | |
| ### Direct Environment Testing | |
| Test the environment logic directly without starting the HTTP server: | |
| ```bash | |
| # From the server directory | |
| python3 server/Container_Yard_environment.py | |
| ``` | |
| This verifies that: | |
| - Environment resets correctly | |
| - Step executes actions properly | |
| - State tracking works | |
| - Rewards are calculated correctly | |
| ### Running Locally | |
| Run the server locally for development: | |
| ```bash | |
| uvicorn server.app:app --reload | |
| ``` | |
| ## Project Structure | |
| ``` | |
| Container_Yard/ | |
| ├── .dockerignore # Docker build exclusions | |
| ├── __init__.py # Module exports | |
| ├── README.md # This file | |
| ├── openenv.yaml # OpenEnv manifest | |
| ├── pyproject.toml # Project metadata and dependencies | |
| ├── uv.lock # Locked dependencies (generated) | |
| ├── client.py # ContainerYardEnv client | |
| ├── models.py # Action and Observation models | |
| └── server/ | |
| ├── __init__.py # Server module exports | |
| ├── Container_Yard_environment.py # Core environment logic | |
| ├── app.py # FastAPI application (HTTP + WebSocket endpoints) | |
| └── Dockerfile # Container image definition | |
| ``` | |