Data Validation

The app validates mapped data against the requirements of the data model at various stages of the mapping process.

Validation includes:

Validation Type	Settings	Description
Table	RequiredColumn	Does the table contain all required columns for the selected table type?
Table	RequiredValue	Does the table contain all required values for the selected table type? A required value means that not only must the column be present, but every row in that column must have a value.
DataType	DataType	Do all values in the column match the required data type (unless blank)? Warning Data type validation is currently incomplete in comparison to data type normalization (Data Normalization), e.g. date and time are not checked. It should be made consistent and check for the same conditions as normalization (without changing the values).
Component	DataType	If the column data type is component, do all the component names match or map to valid component names?
Enumeration	DataType	If the column data type is enum:<enumeration set name>, do all the values match or map to valid values for the enumeration set?
UOM	hasUOM	Are UOMs specified for every value if a column is marked as having UOMs? Are the specified UOM strings valid UOMs? Warning This does not currently check if the UOM is valid for a specific column type. E.g. psia is a valid UOM, but is not valid for a temperature. UOM Dimension could be used to validate this, but it is not currently implemented.

Validation is performed on the column data (table_data_mapped) and header data (table_data_mapped) which are the resulting table and header data after the column and header mapping processes have completed. The results are presented to the user in a dropdown:

If a component cannot be mapped to the standard component list, the user is presented with a dropdown of valid components to map it to. These mappings are recorded and used for all future tables and reports.

Similarly, if an enumeration cannot be mapped to the required enumeration set, the user is presented with a dropdown of valid enumeration values to map it to. These mappings are recorded and used for all future tables and reports.

Warning

UOM validation messages are currently not presented properly in the dropdown.

Note

UOM mapping is done in the Map Columns and Map Headers views, but could be added here as well for consistency.

A table with validations can be saved and merged, but uploading the data to the data api (separate manual step) may fail. It is up to the user to correct validation errors as needed.