Help Centre

Updates
Pipeline Processor versions
The Pipeline Processor determines how your data is processed. Updating to a newer version may change your pipeline results. Test how the changes affect your data by creating a duplicate pipeline and updating the version.

Updating the Pipeline Processor version

The latest Pipeline Processor version is applied to newly created pipelines automatically.
For existing pipelines, an icon appears in the top right indicating that a new version is available. Clicking the update button will re-interpret your uploaded data using the new processor version.
Updating may change your uploaded data and affect your pipeline results. Before updating a pipeline, it's recommended to test how the changes will affect your data by creating a duplicate of the pipeline and updating its processor version.
Datasets linked from the Data Repo will not be automatically updated to the new version. To update a dataset, navigate to the dataset and follow the instructions here.

Version 2

November 2019
Version 2 contains updates in three areas:

1. Interpreting data in .xlsx files

The way data in uploaded .xlsx files is handled has been changed so that values generated by formulae are treated in the same way as regular values. The changes are:
1.
  Columns generated by formulae are now interpreted as values rather than as formatted text.

2.
  Dates created by formulae are now interpreted as ISO rather than as formatted text.

3.
  Results of booleans (e.g. IF statements) are now interpreted ‘true’ / ‘false’ rather than ‘1’/’0’.

4.
  Results of formulae with text in quotes now retain the quotes rather than removing them.
Example
1
2
3
4
Version 1
$10.00
01/10/2019
1
string
Version 2
10
20191001
true
“string”

2. Arithmetic equations treat blank as zero

In Calculate and Validate, arithmetic equations ( + - / * ^ ) and some functions (ABS, ROUND) now interpret blank values as zero. The functions MIN and MAX now ignore non-numeric values.
Previously, if these functions encountered a blank value they would return ‘NaN’ (‘Not a Number’), which could cause problems with downstream processes. Now the functions will always give a valid numeric result when encountering blanks.
Inequalities ( < <= > => ) will still produce a NaN if encountering a blank.
Example
+
MIN
MIN
Version 1
2 + ‘blank’ = NaN
MIN(‘blank’,1) = NaN
MIN(‘blank’,-1) = NaN
Version 2
2 + ‘blank’ = 2
MIN(‘blank’,1) = 1
MIN(‘blank’,-1) = -1

3. Flexible interpretation of column values

In Calculate and Validate a numeric value can be the result of a text function, and vice-versa.
Previously the whole calculation would be ignored with the result column not being created/updated.
Example
Age
IF
Concatenate
Version 1
1
IF('Age'="", "", 'Age' + 1) = Column not created
ABS( CONCATENATE("-", "1")) = Column not created
Version 2
1
IF('Age'="", "", 'Age' + 1) = 2
ABS( CONCATENATE("-", "1")) = 1

Version 1

Original Pipeline Processor version.