Known limitations

This release of the data flow capability includes several limitations and known issues that will be fixed over time.

General

• The sample size for the data preview can be configured, but only up to 10000 rows.

• A minimum supported size for the browser window exists, but is not enforced. Reducing the window size too much will cause a poor experience with the interface.

• If the source or target of a data flow refers to a space that has been renamed since the creation of the flow, it will appear valid but will fail to run. Configure your sources and targets again to fix the data flow.

• The Modified date available in the Overview tab will only be updated after a run or when any of the name, description or tags of the data flow are edited, and not when the design of the data flow itself is changed.

• To make sure latest developments and fixes from the development team are properly applied to your data flow, we recommend making an edit to the name or description of the data flow. This will trigger a recomputation of the script and related apps.

• When editing a node (source, processor, target) in the properties panel, name or description edits in the upper panel, and node configuration in the lower panel must be saved separately or information can be lost. In the example of a processor, if you edit the name without clicking Save changes, change the processor parameters and click Apply, the name change will not be saved.

Data preview

• If you change the type or name of a data flow target, the underlying script will be updated, invalidating the current data preview. You need to manually refresh the data preview panel if you want to use it again.

• The row limit of the data preview can cause some inconsistencies between rows displayed in two processors that seemingly have the same configuration. For example, two Aggregate processors can return different results if one uses a load statement referring directly to the source data file, and the other uses a resident load statement referring to the same table but placed after a processor like Fork This is due to slight differences in the script generation. However, when running the data flow on the full data, the values will be correctly calculated.

Sources and targets

• Data flows targets can only be stored as files.

Processors

• Dates:

  • The Convert to date and Convert to timestamp can fail when applied to a field that is read from a source format with a native date support such as Excel.

  • When adding years, months, weeks or days to a timestamp with the Add duration function, it will be formatted as a date. However, the underlying value contains the correct timestamp.

  • When formatting dates using a Date processor located after a Filter, Fork, or Unpivot processor (where the script uses a resident load statement), you will get a validation error and the function may not work. Try moving the Date processor upstream of the other ones.

• Window: Relies on the window script function, with the same limitations.

• Filter: The processors show "Undefined" as description when no field has been selected.

Collaborative work

• Although collaboration from two users or more on a single data flow is limited at this time, it is possible for multiple users to edit a data flow in a shared space. If you open a data flow, and someone makes a concurrent change, a prompt will open to notify that you need to reload the data flow. After you get the latest version, you can make your change.

• Data flows do not currently support the versioning feature found in Scripts and Apps.