Data Validation
==================

The app validates mapped data against the requirements of the data model at various stages of the mapping process.

Validation includes:

.. list-table::
    :widths: 15 15 40
    :header-rows: 1

    * - **Validation Type**
      - **Settings**
      - **Description**
    
    * - **Table**
      
      - ``RequiredColumn``

      - Does the table contain all required columns for the selected table type?

    * - **Table**

      - ``RequiredValue``  

      - Does the table contain all required values for the selected table type? A required value means that not only must the column be present, but every row in that column must have a value.
    
    * - **DataType**

      - ``DataType``

      - Do all values in the column match the required data type (unless blank)?
        
        .. warning::

            Data type validation is currently incomplete in comparison to data type normalization (:doc:`data_normalization`), e.g. date and time are not checked.
            It should be made consistent and check for the same conditions as normalization (without changing the values).
    
    * - **Component**

      - ``DataType``

      - If the column data type is ``component``, do all the component names match or map to valid component names?
    
    * - **Enumeration**

      - ``DataType``

      - If the column data type is ``enum:<enumeration set name>``, do all the values match or map to valid values for the enumeration set?
    
    * - **UOM**

      - ``hasUOM``  

      - Are UOMs specified for every value if a column is marked as having UOMs?
        
        Are the specified UOM strings valid UOMs?

        .. warning::

            This does not currently check if the UOM is valid for a specific column type.
            E.g. `psia` is a valid UOM, but is not valid for a temperature. 
            UOM Dimension could be used to validate this, but it is not currently implemented.

Validation is performed on the column data (``table_data_mapped``) and header data (``table_data_mapped``) which are the resulting table and header data after the column and header mapping processes have completed. 
The results are presented to the user in a dropdown:

.. image::  ../../../../media/validation_dropdown.png
    :alt: Validation Dropdown Example
    :align: center

If a component cannot be mapped to the standard component list, the user is presented with a dropdown of valid components to map it to. 
These mappings are recorded and used for all future tables and reports.

Similarly, if an enumeration cannot be mapped to the required enumeration set, the user is presented with a dropdown of valid enumeration values to map it to. 
These mappings are recorded and used for all future tables and reports.

.. warning::

  UOM validation messages are currently not presented properly in the dropdown.

.. note::

  UOM mapping is done in the `Map Columns` and `Map Headers` views, but could be added here as well for consistency.

A table with validations can be saved and merged, but uploading the data to the data api (separate manual step) may fail. 
It is up to the user to correct validation errors as needed.