Architecture Overview
This library provides a robust framework for managing real-time data streams and command processing, designed for applications that require low-latency communication and resilient operations. Developers interact with the core API surface, which facilitates the ingestion and management of entities, telemetry, and commands. Key entry points and exports can be found within the main application modules, enabling developers to integrate their systems with the Overwatch platform.
Internally, Constellation Overwatch is orchestrated by an event-driven architecture powered by NATS JetStream for durable messaging. Specialized workers process distinct event types. The Entity Worker maintains the state of system entities by subscribing to relevant NATS streams, persisting changes to a libSQL database, and synchronizing state to the NATS KV store. The Telemetry Worker efficiently handles high-volume telemetry data, updating entity states in the NATS KV store for real-time consumption. The Command Worker manages the reliable delivery and acknowledgment of commands to entities, abstracting communication protocols.
A critical component for maintaining data consistency is the NATS KV Synchronizer. This service ensures that the NATS Key-Value store, used for rapid state synchronization and UI updates, accurately reflects the persistent data in libSQL. This synchronization is crucial for enabling immediate UI updates and low-latency entity state lookups. The architecture also includes an Embedded NATS Server Management module, simplifying deployment by integrating the NATS JetStream server directly into the application, providing a self-contained messaging backbone.
This architecture emphasizes resilience and real-time responsiveness. The use of NATS JetStream ensures messages are not lost, even in the face of network issues or service restarts. The separation of concerns among the workers allows for independent scaling and failure handling. The NATS KV store provides a performant cache for frequently accessed state, minimizing database load and enabling the use of Server-Sent Events for a truly reactive user experience.
For those looking to contribute, understanding the interplay between NATS JetStream and the various workers is paramount. Start by exploring the pkg/workers directory to see how data flows into and out of each specialized worker. The pkg/synchronizer/natskv module is essential for grasping how persistent and real-time states are reconciled. Familiarize yourself with the NATS JetStream documentation and how it's configured and utilized within the pkg/nats module. The core logic for entity and telemetry processing can be found within their respective worker files.