beVault 3.5 - Impact and migration

The version 3.5 of beVault completely reworks the Distribute module. As always, beVault automates the migration process from version 3.4 to version 3.5. However, in some specific cases, the new version can have a small impact on an existing project. This document lists the different impacts and how to adapt your project for this new release.

New script path to reference in States

The path to reference an information mart script changed with the version 3.5. This is due to the fact that beVault now allows creating your own information marts instead of just having a Global section and one per source system. Therefore, the path to reference an information mart script in the orchestrator changed :

version 3.4

Global/[script path]
SourceSystems/[source code]/[script path]

version 3.5

im/[IM name]/[script name].sql

The automatic migration process already does the following things:

Creates one information mart per source system if there was a script in it and one for the previous Global section
In States, for each task “ExecuteDataVaultQuery”, for the parameter QueryName
- replace “Global/” with “im/Global”
- replace “SourceSystems/” with “im/”
- add “.sql” at the end

Example:

JSON

{
  "StartAt": "ref-snapshot",
  "TimeoutSeconds": 720,
  "States": {
    "ref-snapshot": {
      "TimeoutSeconds": 360,
      "Type": "Task",
      "Resource": "ExecuteDataVaultQuery",
      "Parameters": {
        "QueryType": "Reader",
        "QueryName": "Global/ref/snapshots",
        "EnvironmentName": "Production",
        "ProjectName": "default"
      },
      "Next": "default-Production-im"
    }
    "generate-dim": {
      "TimeoutSeconds": 360,
      "Type": "Task",
      "Resource": "ExecuteDataVaultQuery",
      "Parameters": {
        "QueryType": "Reader",
        "QueryName": "SourceSystems/odoo/dim_employee",
        "EnvironmentName": "Production",
        "ProjectName": "default"
      },
      "End": true
    }
  }
}

JSON

{
  "StartAt": "ref-snapshot",
  "TimeoutSeconds": 720,
  "States": {
    "ref-snapshot": {
      "TimeoutSeconds": 360,
      "Type": "Task",
      "Resource": "ExecuteDataVaultQuery",
      "Parameters": {
        "QueryType": "NonQuery",
        "QueryName": "im/Global/ref/snapshots.sql",
        "EnvironmentName": "Production",
        "ProjectName": "default"
      },
      "Next": "default-Production-im"
    }
    "generate-dim": {
      "TimeoutSeconds": 360,
      "Type": "Task",
      "Resource": "ExecuteDataVaultQuery",
      "Parameters": {
        "QueryType": "NonQuery",
        "QueryName": "im/odoo/dim_employee.sql",
        "EnvironmentName": "Production",
        "ProjectName": "default"
      },
      "End": true
    }
  }
}

Limitation

The parameter will be changed only for hardcoded value. This means that if you fetch the QueryName from the input or use a function such as States.Format(), it will not be migrated.

Example:

JSON

"QueryName.$": "States.Format('SourceSystems/odoo/dim_{}',$.dimension)",

Change of database user used for Reader queries

While deploying beVault, you must specify two different users for the servers available for the environments, one read-only user and one read-write user (CFR Metavault Configuration | Configuration ).

Starting version 3.5, the database user used while executing a task “ExecuteDataVaultQuery” with the QueryType set to Reader will be the read-only one. For the NonQuery type, the read-write user will be used instead. This allows more flexibility in the security management of the database.

As most of the queries of the information mart involve deleting and creating views, a read-write user is needed. Therefore, the automatic migration for the version 3.5 changes automatically all QueryTypes to “NonQuery” to ensure that the user used for the query remains the same.

If you used the QueryType reader to properly get a result from an information mart script, you will need to change back manually the QueryType to “Reader”

Distribute module not available

This bug has been fixed in version 3.5.1.

Due to the major migration of the information marts, the dynamic tabs have been altered. If you had opened tabs in the distribute module before the migration, you might have a blank page when you come back to this module for the first time with beVault 3.5.
If this is the case, you can clear your local storage to fix this issue. To do so:

open the developer console (F12 on most browsers) and go to Application (Chrome) or Storage (Firefox)
Expand “Local storage”
Right click on beVault’s URL and then Clear
Refresh the page (F5)