Media Manager for handling camera and audio devices.

This class provides a unified interface for managing both camera and audio devices across different backends. It handles initialization, configuration, and cleanup of media resources.

Audio implementation using GStreamer.

Extends AudioBase with two GStreamer-specific helpers:

• clear_output_buffer(): flush queued playback data without stopping the pipeline (no-op by default; useful before refilling the buffer).
• clear_player(): flush the playback appsrc immediately via GStreamer flush events, dropping any queued audio.

Return the card number of the ReSpeaker sound card, or 0 if not found.

Note: This function runs ‘arecord -l’ to list available audio capture devices and processes the output to find Reachy Mini Audio or ReSpeaker devices. It’s primarily used on Linux systems with ALSA audio configuration.

The function returns:

• Positive integer: Card number of detected Reachy Mini Audio device
• 0: No Reachy Mini Audio device found, using default sound card
• -1: Error occurred while trying to detect audio devices

card_num = get_respeaker_card_number()
if card_num > 0:
    print(f"Using Reachy Mini Audio card {card_num}")
elif card_num == 0:
    print("Using default sound card")
else:
    print("Error detecting audio devices")

Check if ~/.asoundrc exists and contains both reachymini_audio_sink and reachymini_audio_src.

Note: This function checks for the presence of the ALSA configuration file ~/.asoundrc and verifies that it contains the necessary configuration entries for Reachy Mini audio devices (reachymini_audio_sink and reachymini_audio_src). These entries are required for proper audio routing and device management.

if has_reachymini_asoundrc():
    print("Reachy Mini audio configuration is properly set up")
else:
    print("Need to configure Reachy Mini audio devices")
    write_asoundrc_to_home()  # Create the configuration

Check if ~/.asoundrc exists and is correctly configured for Reachy Mini Audio.

Write the .asoundrc file with Reachy Mini audio configuration to the user’s home directory.

This function creates an ALSA configuration file (.asoundrc) in the user’s home directory that configures the ReSpeaker sound card for proper audio routing and multi-client support. The configuration enables simultaneous audio input and output access, which is essential for the Reachy Mini Wireless version’s audio functionality.

The generated configuration includes:

• Default audio device settings pointing to the ReSpeaker sound card
• dmix plugin for multi-client audio output (reachymini_audio_sink)
• dsnoop plugin for multi-client audio input (reachymini_audio_src)
• Proper buffer and sample rate settings for optimal performance

Note: This function automatically detects the ReSpeaker card number and creates a configuration tailored to the detected hardware. It is primarily used for the Reachy Mini Wireless version.

The configuration file will be created at ~/.asoundrc and will overwrite any existing file with the same name. Existing audio configurations should be backed up before calling this function.

Class to interface with the ReSpeaker XVF3800 USB device.

Find and return the ReSpeaker USB device with the given Vendor ID and Product ID.

Note: This function searches for USB devices with the specified Vendor ID and Product ID using libusb backend. The default values target XMOS XVF3800 devices used in ReSpeaker microphone arrays.

from reachy_mini.media.audio_control_utils import find

# Find default ReSpeaker device
respeaker = find()
if respeaker is not None:
    print("Found ReSpeaker device")
    respeaker.close()

# Find specific device
custom_device = find(vid=0x1234, pid=0x5678)

Initialize the ReSpeaker USB device. Looks for both new and beta device IDs.

Note: This function attempts to initialize a ReSpeaker microphone array by searching for USB devices with known Vendor and Product IDs. It tries:

1. New Reachy Mini Audio firmware (0x38FB:0x1001) - preferred
2. Old ReSpeaker firmware (0x2886:0x001A) - with warning to update

The function handles USB backend errors gracefully and returns None if no compatible device is found or if initialization fails.

from reachy_mini.media.audio_control_utils import init_respeaker_usb

# Initialize ReSpeaker device
respeaker = init_respeaker_usb()
if respeaker is not None:
    print("ReSpeaker initialized successfully")
    # Use the device...
    doa = respeaker.read("DOA_VALUE_RADIANS")
    respeaker.close()
else:
    print("No ReSpeaker device found")

Camera that reads BGR frames from the daemon’s local IPC endpoint.

The WebRTC daemon exposes BGR camera frames via a local IPC mechanism:

• Linux / macOS: unixfdsink / unixfdsrc (Unix domain socket)
• Windows: win32ipcvideosink / win32ipcvideosrc (shared memory)

Since the daemon’s IPC branch already converts to BGR, the reader pipeline is simply source → queue → appsink with no extra conversion.

Undistort a single pixel coordinate to normalized camera coordinates.

Pure numpy equivalent of cv2.undistortPoints(). Supports the OpenCV distortion model with up to 12 coefficients (rational model + thin prism): D = (k1, k2, p1, p2, k3, k4, k5, k6, s1, s2, s3, s4)

Also works with 5-coefficient models (k1, k2, p1, p2, k3) and zero-distortion.

The algorithm matches OpenCV’s cvUndistortPointsInternal:

1. Remove camera intrinsics to get normalized distorted coordinates.
2. Iteratively solve for undistorted coordinates using a damped fixed-point method with adaptive step size.

Reference: OpenCV distortion model and undistortPoints algorithm: https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html https://github.com/opencv/opencv/blob/4.x/modules/calib3d/src/undistort.dispatch.cpp

Scale camera intrinsics for a different resolution with cropping.

Base class for camera resolutions.

Enumeration of standardized camera resolutions and frame rates supported by Reachy Mini cameras. Each enum value contains a tuple of (width, height, fps).

Note: The enum values are tuples containing (width, height, frames_per_second, crop_factor). Not all resolutions are supported by all camera models - check the specific camera specifications for available resolutions.

from reachy_mini.media.camera_constants import CameraResolution

# Get resolution information
res = CameraResolution.R1280x720at30fps
width, height, fps, crop_factor = res.value
print(f"Resolution: {width}x{height}@{fps}fps")

# Check if a resolution is supported by a camera
from reachy_mini.media.camera_constants import ReachyMiniLiteCamSpecs
res = CameraResolution.R1920x1080at60fps
if res in ReachyMiniLiteCamSpecs.available_resolutions:
    print("This resolution is supported")

Base camera specifications.

Dataclass containing specifications for a camera model, including supported resolutions, calibration parameters, and USB identification information.

Note: The intrinsic matrix K has the format: [[fx, 0, cx], [ 0, fy, cy], [ 0, 0, 1]]

Where fx, fy are focal lengths in pixels, and cx, cy are the principal point coordinates (typically near the image center).

from reachy_mini.media.camera_constants import CameraSpecs

# Create a custom camera specification
custom_specs = CameraSpecs(
    name="custom_camera",
    available_resolutions=[CameraResolution.R1280x720at30fps],
    default_resolution=CameraResolution.R1280x720at30fps,
    vid=0x1234,
    pid=0x5678,
    K=np.array([[800, 0, 640], [0, 800, 360], [0, 0, 1]]),
    D=np.zeros(5)
)

Arducam camera specifications.

Reachy Mini Lite camera specifications.

Reachy Mini Wireless camera specifications.

Older Raspberry Pi camera specifications. Keeping for compatibility.

Mujoco simulated camera specifications.

Generic webcam specifications (fallback for any webcam).

WebRTC client that provides both camera frames and audio.

Implements the same public API surface as GStreamerCamera (for video) and GStreamerAudio (for audio) so that MediaManager can assign the same instance to both its camera and audio slots.

Daemon-side GStreamer media server.

Owns the camera and audio hardware and distributes media to consumers:

• IPC branch — raw BGR frames via unixfdsink / win32ipcvideosink for on-device applications (GStreamerCamera reads from this).
• WebRTC branch — encoded video + audio via webrtcsink for remote clients (GstWebRTCClient connects to this).
• Sound playback — playbin for playing WAV files on the speaker.