You are here:
Dataflow Conversion Mapping
During conversion, dataflow functionality is mapped to similar recipe functionality. Consider the mapping specifications when evaluating the conversion of your dataflows to recipes.
Dataflow Transformations
At a glance dataflow to recipe conversion mapping.
| Dataflow | Conversion | Recipe |
|---|---|---|
| append |
|
Append |
| augment |
|
Join |
| computeExpression |
|
Transform (Custom formula) |
| computeRelative |
see parameters |
Transform (Custom formula with multiple rows) |
| delta | ||
| digest |
|
Input |
| dim2mea |
|
Transform (Dimension to measure) |
| edgemart |
|
Input |
| export |
see parameters |
Output (as .csv) |
| filter |
|
Filter |
| flatten |
|
Transform (Flatten) |
| prediction |
|
Discovery Predict |
| sfdcDigest |
see parameters |
Input and Transform |
| sfdcRegister |
|
Output |
| sliceDataset |
|
Transform (Drop columns) |
| update |
|
Update |
| Dataflow Parameter | Conversion | Supported in Recipe |
|---|---|---|
| sources |
|
Append |
| enableDisjointedSchemaMerge |
|
Append (Allow schema merge) |
| Dataflow Parameter | Conversion | Supported in Recipe |
|---|---|---|
| operation |
|
Join (lookup) |
| left |
|
Join (lookup) |
| left_key |
|
Join (lookup) |
| right |
|
Join (lookup) |
| relationship |
|
Join (lookup) |
| right_select |
|
Join (lookup) |
| right_key |
|
Join (lookup) |
| Dataflow Parameter | Conversion | Supported in Recipe |
|---|---|---|
| mergeWithSource |
|
Transform (Drop columns) |
| saqlExpression |
|
Transform (SQL in custom formula) |
| format |
|
Transform (Custom formula) |
| fiscalMonthOffset | Use Data Prep Date Configurations |
|
| isYearEndFiscalYear | Use Data Prep Date Configurations |
|
| firstDayOfWeek | Use Data Prep Date Configurations |
|
| precision |
|
Transform (Custom formula) |
| scale |
|
Transform (Custom formula) |
| defaultValue |
|
Transform (Custom formula) The defaultValue in dataflows support function
calls. For example, |
| Dataflow Parameter | Conversion | Supported in Recipe |
|---|---|---|
| partitionBy |
|
Transform (Custom formula with multiple rows) |
| orderBy |
|
Transform (Custom formula with multiple rows) |
| computedFields.name |
|
Transform (Custom formula with multiple rows) |
| computedFields.label |
|
Transform (Custom formula with multiple rows) |
| computedFields.description | ||
| computedFields.expression.sourceField |
|
Transform (Custom formula with multiple rows) |
| computedFields.expression.offset |
|
Transform (Custom formula with multiple rows) |
| computedFields.expression.default |
|
Transform (Custom formula with multiple rows) |
| computedFields.expression.saqlExpression |
|
Transform (Custom formula with SQL) Self-referencing formulas don’t convert and aren’t supported. For example, using computeRelative to create a field “MyNewField”, which has SAQL expression referencing previous(MyNewField). Achieve similar results using recipes. For example, if using computeRelative in dataflow to:
|
| computedFields.expression.type |
|
Transform (Custom formula with multiple rows) |
| computedFields.expression.scale |
|
Transform (Custom formula with multiple rows) |
Delta transform isn’t supported in Recipes. To calculate changes over time, consider using a transformation node with windowing functions in multiple row custom formulas.
| Dataflow Parameter | Conversion | Supported in Recipe |
|---|---|---|
| connectionName |
|
Input |
| object |
|
Input |
| fields |
|
Input |
| Dataflow Parameter | Conversion | Supported in Recipe |
|---|---|---|
| measureDefault |
|
Transform (Dimension to measure) |
| measureType |
To extract all columns in a dataset, use the Include All Fields option in the Input node.
| Dataflow Parameter | Conversion | Supported in Recipe |
|---|---|---|
| alias |
|
Input |
| Dataflow Parameter | Conversion | Supported in Recipe |
|---|---|---|
| source |
|
Output (to .csv) |
| target | ||
| userId |
| Dataflow Parameter | Conversion | Supported in Recipe |
|---|---|---|
| filter |
|
Filter |
| saqlFilter |
|
Filter (SQL filter expression) SAQL Use recipe Transformation custom formula with |
| Dataflow Parameter | Conversion | Supported in Recipe |
|---|---|---|
| include_self_id |
|
Transform (Flatten) |
| self_field |
|
Transform (Flatten) |
| parent_field |
|
Transform (Flatten) |
| multi_field |
|
Transform (Flatten) |
| path_field |
|
Transform (Flatten) |
| Dataflow Parameter | Conversion | Supported in Recipe |
|---|---|---|
| predictionDefinitionName |
|
Discovery Predict |
| predictionColumnName |
|
Discovery Predict |
| predictionColumnLabel |
|
Discovery Predict |
| Dataflow Parameter | Conversion | Supported in Recipe |
|---|---|---|
| object |
|
Input |
| incremental | ||
| fullRefreshToken | ||
| fields |
|
Input |
| filterConditions |
|
Filter (Filter logic) |
| complexFilterConditions | Use Filter (SQL expression) |
| Dataflow Field Attributes | Conversion | Supported in Recipe |
|---|---|---|
| name |
|
Input |
| defaultValue | Use Transform (Custom formula with case or
coalesce) |
|
| type | Use Transform (Replace or Custom formula) | |
| fiscalMonthOffset | Use Data Prep Date Configurations |
|
| isYearEndFiscalYear | Use Data Prep Date Configurations |
|
| firstDayOfWeek | Data Prep Date Configurations |
|
| isMultiValue | Use Transform (Custom formula multivalue functions) |
|
| multiValueSeperator | Use Transform (Custom formula multivalue functions) |
|
| precision | Use Transform (Edit attributes) |
|
| scale | Use Transform (Edit attributes) |
| Dataflow Parameter | Conversion | Supported in Recipe |
|---|---|---|
| alias |
|
Output |
| name |
|
Output |
| source |
|
Output |
| rowLevelSharingSource |
|
Output |
| rowLevelSecurityFilter |
|
Output |
| Dataflow Parameter | Conversion | Supported in Recipe |
|---|---|---|
| source |
|
Transform (Drop columns) |
| mode |
|
Transform (Drop columns) |
| fields |
|
Transform (Drop columns) |
| Dataflow Parameter | Conversion | Supported in Recipe |
|---|---|---|
| left |
|
Update |
| right |
|
Update |
| left_key |
|
Update |
| right_key |
|
Update |
| update_columns |
|
Update |
Dataflow SAQL
Dataflows use SAQL and recipes use SQL. During conversion, SAQL is changed to SQL whenever possible.
In
recipes, try using simple operators with Dates or DateTimes such as +,-,=,<,>,<=,>=. For example, add hours by using
(DateTime) + (TimeInterval).
Below are instances when SAQL is not automatically converted to SQL, and the recommended SQL alternative. In recipes, use SQL in a Transform custom formula.
| Dataflow (SAQL) | Recipe (SQL) |
|---|---|
| DateField_sec_epoch | to_unix_timestamp(DateField) |
| DateField_day_epoch | to_unix_timestamp(DateField) / (60 * 60 * 24) |
| DateField_Year | format_number(year(DateField), '0000') |
| DateField_Quarter | format_number(quarter(DateField), ‘0') |
| DateField_Month | format_number(month(DateField), '00') |
| DateField_Week | format_number(weekofyear(DateField), '00') |
| DateField_Day | format_number(day(DateField), '00') |
| DateField_Hour | format_number(hour(DateField), '00') |
| DateField_Minute | format_number(minute(DateField), '00') |
| DateField_Second | format_number(second(DateField), '00') |
| toDate() | to_timestamp() |
| date_to_epoch() | to_unix_timestamp() |
| month_first_day(toDate('CloseDate_sec_epoch')) | date_trunc('month', CloseDate) |
| month_last_day(toDate('CloseDate_sec_epoch')) | date_trunc('month', CloseDate) + INTERVAL 1 month - INTERVAL 1 day |
week_first_day(toDate('CloseDate_sec_epoch')) This function always returns Sunday of the week |
date_trunc('week', CloseDate) This function always returns Monday of the week |
| Other first_day or last_day functions | Use date_trunc on the different parts: 'year', 'quarter', 'month', 'week'. |
| toString(toDate('CloseDate_sec_epoch'), "yyyy-MM-dd") | date_format(CloseDate, 'yyyy-MM-dd') |
| date_to_string() | date_format(CloseDate, 'yyyy-MM-dd') |
| date_diff("year", toDate(StartDate_sec_epoch), toDate(EndDate_sec_epoch)) | year(EndDate) - year(StartDate) |
| date_diff("month", toDate(StartDate_sec_epoch), toDate(EndDate_sec_epoch)) | months_between(EndDate, StartDate) |
| date_diff("quarter", toDate(StartDate_sec_epoch), toDate(EndDate_sec_epoch)) | (year(EndDate)-year(StartDate))*4 + quarter(EndDate) - quarter(StartDate) |
| date_diff("week", toDate(StartDate_sec_epoch), toDate(EndDate_sec_epoch)) | floor(datediff(EndDate, StartDate) / 7) |
| date_diff("day", toDate(StartDate_sec_epoch), toDate(EndDate_sec_epoch)) | datediff(EndDate, StartDate) |
| date_diff("hour", toDate(StartDate_sec_epoch), toDate(EndDate_sec_epoch)) | (to_unix_timestamp(EndDate) - to_unix_timestamp(StartDate))/(60*60) |
| date_diff("minute", toDate(StartDate_sec_epoch), toDate(EndDate_sec_epoch)) | (to_unix_timestamp(EndDate) - to_unix_timestamp(StartDate))/(60) |
| date_diff("second", toDate(StartDate_sec_epoch), toDate(EndDate_sec_epoch)) | to_unix_timestamp(EndDate) - to_unix_timestamp(StartDate) |



