Header Mapping¶

Header mapping is the process of identifying or entering individual keyword values for the selected test type that are not part of the test’s tabular data.

Object Model¶

The following data fields are used for header mapping:

Warning

An object model is defined for header mapping, but it is not currently used. Equivalent arrays are used instead. Future work should prioritize refactoring to use the object model.

Header Mapping Object Model¶
Field	Description	Shown as
`field_name`	Name of the field in the standard data model.	not shown
`predicted_value`	Value that is predicted for the field based on a variety of methods, including: Extracting keywords based on Keyword Mapping. Extracting saturation and reservoir pressure values from the test’s tabular data. Extracting test temperature from text in the table’s raw data (`table_data_raw`) via pattern matching. Extracting sample ID and container ID by matching text in the table’s raw data to previously identified samples in the report. Inheriting sample, container, and test IDs from previous tables.	Shown as ✨ beside the Report Value field, if the report value field matches the predicted value.
`value`	Final value for the field, either populated automatically or entered by the user.	Report Value
`uom`	Final value for the UOM (unit of measure) for the field, either populated automatically or entered by the user.	Report UOM
`predicted_uom`	UOM that is predicted for the field along with the predicted value.	Shown as ✨ beside the Report UOM field, if the report UOM field matches the predicted UOM.
`mapped_uom`	UOM that is mapped to the field in the standard data model, either populated automatically or manually selected from a dropdown list.	Standard UOM
`std_uom`	Standard model UOM equivalent of the report UOM.	Shown as ✨ beside the Standard UOM field, if the mapped UOM matches the standard UOM. Also shown as ✨ beside the standard UOM in the standard UOM dropdown list.
`has_uom`	Bool: Should the field have an associated UOM? Populated from the `uoms` collection in the configuration cache.	Controls whether UOM fields are shown.
`uom_dimension`	Dimension-associated UOM, populated from the `table_columns` collection in the configuration cache.	Controls what UOMs are displayed in the Standard UOM dropdown list.
`default_uom`	Default UOM to use if none are specified/predicted. Populated from the `table_columns` collection in the configuration cache.	NA

Saturation and Reservoir Pressure Extraction¶

Tests with multiple test steps at different pressures such as CCE and DL typically indicate which rows (pressures) represent saturation and reservoir pressures, either by embedding keywords in the value itself (e.g. 3000 Psat) or (Pres=4500) or putting keywords in otherwise empty columns.

The extraction process tries to match a variety of different terms for those pressures to text in the rows or the values. If a term matches then that row’s stepPressure is assigned to the saturation or reservoir pressure field and the keyword text is removed.

Test Temperature Extraction¶

For tables that have testTemperature fields, the test temperature is typically listed in the table heading (e.g. CCE test @ 270F). The extraction process searches all of the text in the raw table for patterns that match temperature units of measure and extracts them as the test temperature. Note that it searches the raw data (table_data_raw) since headings would have been removed from table_data_edited during the template matching and editing process.

Warning

This matching process is not very rigorous, and can pick up incorrect values. An improvement would be to search for more specific patterns/phrases (e.g. CCE test @ 270F) and only extract the temperature if the phrase matches.

Sample and Container ID Matching¶

If a test has sample and container id fields and samples have previously been recorded for the test, the process will search for each container id within the page’s raw data. If it finds a match then it updates the test’s sample and container IDs to the associated sample values

Inheriting Fields¶

Some tests have multiple parts, such as DL which has DLTest/DLTestSteps (the primary table) and DL composition, DL resid composition, etc. (the secondary tables). These are usually reported on separate pages, but sample ID, container ID, and test number are not repeated on each page.

Because these are required to link the different test parts together, the process will update these fields in secondary tables by locating the previous primary table and copying its values.

Map Header View¶

The Map Header view lets the user enter standard keyword values for the selected table type. Automatic mapping is performed when the view is called up to default the mapping to predicted values, unless the mapping has been previously saved (part of saving the report). This gives the user default predicted values as a starting point.

The user is presented with a mapping table listing every possible header field for the selected table type, with these columns:

Standard Property: The field name of the standard model field.
Report Value: The value of the field extracted from the report’s raw data, or entered by the user.
Report UOM: The unit of measure for the report value, if applicable.
Standard UOM: A dropdown list of possible UOMs for the field, based on the UOM dimension of the field. Will be automatically populated if there is a standard_UOM mapping.

If the field is an enumeration type, a dropdown list is presented in the Report UOM column:

The user can select the desired standard value of the enum from the list and save the mapping. This will be used to predict the enum value in future reports.

If the enum chosen by the user is different from the predicted value, an option to update the enum mapping is presented.

If the field is either sampleID or containerID, a dropdown list is presented in the Report UOM column:

The user can pick from sample/container IDs for any samples that have already been identified in the report.

If the user makes a selection, the corresponding fluidSampleID or fluidSampleContainerID is updated. Otherwise, the field is left blank and the user can enter a value manually.

If the desired sample does not appear in the list, the user can select the Manage Samples button in the Standard UOM column to create it.