AI OS Design

After exploring the features of AI OS, let’s dive into the details of its design.

How agent queries are decomposed into AIOS system calls and how AIOS system calls are dispatched and scheduled.

The AIOS architecture is divided into three layers: the application layer, the kernel layer, and the hardware layer. Each layer has a clear separation of concerns, aiming to simplify the complexity of the system. Higher-level applications interact with the underlying layers through well-defined interfaces such as Software Development Kits (SDKs) and system calls.

1. Application Layer:

   • At the application layer, agent applications are developed using the AIOS-Agent SDK. The SDK provides an interface for requesting system resources through system calls.

   • The SDK allows agents to offload the burden of resource management and ensures that system resources are not directly manipulated by agents through isolation mechanisms.

   • The SDK supports both native agents, developed using AIOS SDK functions, and non-native agents created with different frameworks (e.g., ReAct, Reflexion, Autogen, etc.). The adapter function allows non-native agents to interact with the AIOS kernel resources.

   • For native agent development, the SDK simplifies agent creation by offering predefined modules and APIs, allowing developers to focus on the agent’s core workflows and logic instead of low-level implementation details.

2. Kernel Layer:

   • The kernel layer consists of two components: the traditional OS kernel and the specialized AIOS kernel, each fulfilling distinct roles within the system.

   • The OS kernel manages non-LLM-related computing tasks, while the AIOS kernel is the core innovation of this system.

   • Within the AIOS kernel, several modules handle agent requests. A scheduler dispatches system calls to appropriate modules, and the AIOS kernel integrates multiple LLM endpoints via a unified interface that treats LLMs as cores, similar to CPU cores.

   • A context manager is introduced to support context switching for LLMs, enabling context snapshots and restoration.

   • A memory manager handles agent memory operations, while a storage manager takes care of persistent storage operations.

   • A tool manager loads tools and manages tool call conflicts for the tools supported in the AIOS-Agent SDK.

   • An access manager enforces access control and user intervention, ensuring secure operations.

3. Hardware Layer:

   • The hardware layer consists of physical components like the CPU, GPU, memory, disk, and peripherals. The AIOS kernel does not directly interact with hardware but relies on OS system calls to access hardware resources.