A data package is a logical bundle of data coming from a specific source system, typically representing a coherent business dataset (e.g., “Employees”, “Invoices”). A single package can contain multiple staging tables that land and structure that dataset for ingestion.
After creating a source system, you can create one or more data packages that are linked to the source. Some source systems provide data in separate batches, and the data delivery schedule for these batches can be defined using data packages.
In order to display the data packages, click the button ‘Data packages’ under the column with the same name for the source system that you wish to explore and/or edit.
The following buttons are available for managing data packages:
-
Create a new data package
-
Modify an existing data package
-
Delete an existing data package
Create a Data Package
To create a new data package linked to a source system, click the “Create a new Data Package” button (refer to the instructions for managing data packages).
A pop-up wizard will appear, requiring you to enter several required and optional attributes.
Specifically, you must provide:
-
Name: mandatory — The name of the data package. This name must be unique within the source system, meaning that a single source system cannot have two data packages with the same name.
-
Delivery Schedule: optional — A description of when the data is provided to the Data Vault in text format.
-
Technical Description: optional — A description of the data package in text format.
-
Business Description: optional — Detailed information about the data package from a business perspective.
-
Refresh Type: optional — A description of the frequency with which the content of the data package is refreshed.
-
Format Info: optional — A description of the format of the file that will be sent to the Data Vault, such as CSV or XML, in text format.
-
Expected Quality: optional — A description of the quality of the data package.
After filling out the required and optional attributes for the data package, click the “Save” button to create it. If the data package is successfully created, it will appear in the list of data packages. You can then proceed to manage staging tables (see the instructions for “Staging Tables”).
Modify an Existing Data Package
To modify an existing data package, click the “Open in a new tab” button for the corresponding data package. The process is similar to creating a new data package, with the same fields and rules. The fields that are already set will be pre-filled (refer to the instructions for creating a data package for more information).
A new tab in the module will appear, allowing you to edit the required and optional attributes for the data package (refer to see the “Create a Data Package” subsection for more information).
Once you have made your changes, click the “Save” button to apply them.
Please know that, by clicking on the second button on the vertical menu on the left, a list of the staging tables created under the package is displayed with the possibility to create, display in a new tab and edit or delete a staging table
Delete a Data Package
To delete a data package, click the ‘Remove’ button with the trash icon on the corresponding line. Please note that you cannot delete a data package if it has associated tables.
