This reference guide documents the Workers and Stores used by beVault's States orchestrator. (CFR Architecture & Installation )
Workers are software components that function as specialized web services. They are called by States to execute precise, atomic tasks in your data processing workflows: reading a CSV file, executing a SQL query in a database, making an HTTP call, compressing archives, moving files, and more. Workers regularly poll the States orchestrator to check if they have work to do. When a task is assigned to them, they execute it with the provided parameters and return the result to States.
To accomplish their tasks, Workers access Stores. A Store is an abstract physical endpoint that allows Workers to extract or send data to external systems. There are two categories of Stores:
-
dbStores: for connecting to databases (PostgreSQL, SQL Server, Snowflake, Oracle, MySQL, etc.)
-
fileStores: for connecting to file systems and storage services (local directories, SFTP, FTP, S3, GitLab, etc.)
Stores can be configured in two ways: centrally via the Data Stores module in the States interface, or locally on each Worker's server via a stores.json file or environment variables. When both exist for the same store name, the local configuration takes priority.