9. Automate your dataFactory

If you followed the previous steps, you should have a data model in your database, as well as some state machines in States. Everything has been automatically generated based on your actions in beVault product.

If you look closely at the tables generated in the schema “dv”, you will see that they are all almost empty. This is because we still didn’t execute the workflows to load the data from the staging to the data vault.

In this section, we will see how to execute the state machines to load the data in the data vault and run the data quality tests that we previously created.

1. First, log in to States and go to “State Machines”
   

   

2. The state machines generated to load data from the staging are named [project id]-[environment]-[staging table]. In this case, you should have at least a state machine named tutorial-modeling-crm_cars. Click on the ‘open’ button next to that state machine.
   

   

3. This will open the state machine. To start the execution and load the data in the data vault, click on “Execute”.
   

   

4. Enter “{}” as Execution Input and click on “Execute state machine”
   

   

   
   

5. After a few seconds, you should see that the different steps succeeded and are green.
   

   

6. Repeat this operation with the state machine “tutorial-modeling-dq” to execute the data quality tests. The state machines for the data quality controls are named [project id]-[environment]-[snapshot name] and contain all the DQ tests related to a specific snapshot.

7. Repeat this operation with the other state machine to load all the data packages and feed your entire model with data

If you return to your tables, you will see that you now have some data in it!

If you want to automate the extraction of data directly from your sources, you can take a look at the documentation of States to create your own state machines. States User Reference Guide