Workflow action
Data transform
ut_data
Select by JSON path, map, filter, sort, group and aggregate, pivot, de-duplicate and join lists of records, and convert CSV to JSON and back.
About
What it does
Data transform reshapes lists of records inside a workflow, the work that otherwise takes loops and a chain of collection nodes. Pick values out of an API response with a JSON path (order.lines[*].sku), rename and nest fields, filter with conditions such as status = paid and amount >= 500, sort on several keys, group by one or more fields with count, sum, average, minimum, maximum and distinct counts, pivot values into columns or unpivot columns into rows, and remove duplicates by chosen fields.
Join combines two lists on matching keys (inner, left, right or full), for example orders with the managers of their regions, and reports what did not match. CSV to JSON parses spreadsheet exports to RFC 4180 (quoted fields, embedded commas and line breaks, header row, number and boolean inference) and JSON to CSV writes lists back out with chosen columns, CRLF line endings and protection against formula injection. Everything runs in the sandbox with no network or data access.
Good for
- Total and average order values per region before a weekly summary email
- Join API results with a data table of owners and keep the unmatched rows for review
- Import a CSV export, filter it and write a clean CSV back out
After installing
- Install the package: the action appears in Extensibility → Workflow actions and in the workflow designer's palette under Custom actions.
- Drag Data transform onto a workflow, choose the function and map Items to a list (a data table query result, an API response path or a JSON variable).
- Chain functions: map the {{prefix_items}} output of one step into the Items of the next, e.g. Filter, then Group & aggregate.
- Try the included "Sales by region (sample)" workflow from the API or its start form: it filters paid orders, groups them by region and writes the totals as CSV.
Examples
Real inputs, real outputs
Each example ran in the sandbox when this gallery was built; the outputs are exactly what the step returns.
Totals per region group
Inputs
{
"items": [
{
"id": "SO-1",
"region": "North",
"product": "Desk",
"status": "paid",
"amount": 1200,
"qty": 2
},
{
"id": "SO-2",
"region": "South",
"product": "Chair",
"status": "paid",
"amount": 300,
"qty": 3
},
{
"id": "SO-3",
"region": "North",
"product": "Chair",
"status": "open",
"amount": 450,
"qty": 5
},
{
"id": "SO-4",
"region": "North",
"product": "Desk",
"status": "paid",
"amount": 600,
"qty": 1
},
{
"id": "SO-5",
"region": "South",
"product": "Lamp",
"status": "void",
"amount": 80,
"qty": 2
}
],
"group_by": [
"region"
],
"aggregates": [
"sum:amount",
"avg:amount",
"count_distinct:product"
]
}Outputs
{
"groups": [
{
"region": "North",
"count": 3,
"sum_amount": 2250,
"avg_amount": 750,
"count_distinct_product": 2
},
{
"region": "South",
"count": 2,
"sum_amount": 380,
"avg_amount": 190,
"count_distinct_product": 2
}
],
"count": 2
}Paid orders of 500 or more filter
Inputs
{
"items": [
{
"id": "SO-1",
"region": "North",
"product": "Desk",
"status": "paid",
"amount": 1200,
"qty": 2
},
{
"id": "SO-2",
"region": "South",
"product": "Chair",
"status": "paid",
"amount": 300,
"qty": 3
},
{
"id": "SO-3",
"region": "North",
"product": "Chair",
"status": "open",
"amount": 450,
"qty": 5
},
{
"id": "SO-4",
"region": "North",
"product": "Desk",
"status": "paid",
"amount": 600,
"qty": 1
},
{
"id": "SO-5",
"region": "South",
"product": "Lamp",
"status": "void",
"amount": 80,
"qty": 2
}
],
"conditions": [
{
"field": "status",
"operator": "eq",
"value": "paid"
},
{
"field": "amount",
"operator": "gte",
"value": 500
}
]
}Outputs
{
"items": [
{
"id": "SO-1",
"region": "North",
"product": "Desk",
"status": "paid",
"amount": 1200,
"qty": 2
},
{
"id": "SO-4",
"region": "North",
"product": "Desk",
"status": "paid",
"amount": 600,
"qty": 1
}
],
"count": 2,
"removed": 3
}Products as columns pivot
Inputs
{
"items": [
{
"id": "SO-1",
"region": "North",
"product": "Desk",
"status": "paid",
"amount": 1200,
"qty": 2
},
{
"id": "SO-2",
"region": "South",
"product": "Chair",
"status": "paid",
"amount": 300,
"qty": 3
},
{
"id": "SO-3",
"region": "North",
"product": "Chair",
"status": "open",
"amount": 450,
"qty": 5
},
{
"id": "SO-4",
"region": "North",
"product": "Desk",
"status": "paid",
"amount": 600,
"qty": 1
},
{
"id": "SO-5",
"region": "South",
"product": "Lamp",
"status": "void",
"amount": 80,
"qty": 2
}
],
"rows": [
"region"
],
"column": "product",
"value": "amount",
"fill": 0
}Outputs
{
"items": [
{
"region": "North",
"Desk": 1800,
"Chair": 450,
"Lamp": 0
},
{
"region": "South",
"Desk": 0,
"Chair": 300,
"Lamp": 80
}
],
"columns": [
"Desk",
"Chair",
"Lamp"
],
"count": 2
}Orders with their region manager join
Inputs
{
"left": [
{
"id": "SO-1",
"region": "North",
"product": "Desk",
"status": "paid",
"amount": 1200,
"qty": 2
},
{
"id": "SO-2",
"region": "South",
"product": "Chair",
"status": "paid",
"amount": 300,
"qty": 3
},
{
"id": "SO-3",
"region": "North",
"product": "Chair",
"status": "open",
"amount": 450,
"qty": 5
}
],
"right": [
{
"region": "North",
"manager": "Aroha"
},
{
"region": "South",
"manager": "Ben"
}
],
"left_key": [
"region"
]
}Outputs
{
"items": [
{
"id": "SO-1",
"region": "North",
"product": "Desk",
"status": "paid",
"amount": 1200,
"qty": 2,
"manager": "Aroha"
},
{
"id": "SO-2",
"region": "South",
"product": "Chair",
"status": "paid",
"amount": 300,
"qty": 3,
"manager": "Ben"
},
{
"id": "SO-3",
"region": "North",
"product": "Chair",
"status": "open",
"amount": 450,
"qty": 5,
"manager": "Aroha"
}
],
"count": 3,
"unmatched_left": 0,
"unmatched_right": 0
}Parse a CSV export csv_to_json
Inputs
{
"csv": "id,name,amount\n1,\"Smith, Jo\",12.50\n2,Ana,7"
}Outputs
{
"items": [
{
"id": 1,
"name": "Smith, Jo",
"amount": 12.5
},
{
"id": 2,
"name": "Ana",
"amount": 7
}
],
"columns": [
"id",
"name",
"amount"
],
"count": 2
}Reference
Functions, inputs and outputs
The designer generates each function's input panel from these declarations; outputs appear in the data picker with their types.
Select (JSON path) select
Read a value from an object or list by path: a.b[0].c, items[-1] for the last item, items[*].name for every item's name (wildcards return a list).
Inputs
datajson · required — The object or list to read from.pathstring · required — For example order.lines[0].sku or order.lines[*].sku ($. prefix optional).
Outputs
valuejson — The value at the path (a list when the path has a wildcard; empty when not found).valueslist of jsons — Every value found, as a list.foundbooleancountinteger — How many values were found.
Map fields map
Pick, rename and nest fields of every item: "customer.name as customer", "total" or { "from": "a.b", "to": "c", "default": 0 }.
Inputs
itemslist of objects · required — A list of objects (rows), e.g. a data table query result or a JSON array.fieldslist · required — One per field: source path, optionally "as" a new name (a dotted name creates nested objects), or objects { from, to, default }.keep_other_fieldsboolean — Keep fields not listed (renamed ones move).
Outputs
itemslist of objects — The mapped items.countinteger — How many items were returned.
Filter filter
Keep the items that match conditions: eq, ne, gt, gte, lt, lte (numbers and dates compare as such), contains, not_contains, starts_with, ends_with, in, not_in, empty, not_empty; all must match or any.
Inputs
itemslist of objects · required — A list of objects (rows), e.g. a data table query result or a JSON array.conditionslist of objects · required — List of { "field": "status", "operator": "eq", "value": "paid" }; in / not_in take a list or comma-separated text.each: field (string), operator (string), value (json)combinechoice · all / any — Whether every condition or any condition must match.case_sensitiveboolean — Compare text exactly (off: ignore case).
Outputs
itemslist of objects — The matching items, in their original order.countinteger — How many items matched.removedinteger — How many items did not match.
Sort sort
Sort by one or more keys, ascending or descending, comparing as numbers, text or dates (auto: numbers when both values are numeric). Stable; empty values sort last.
Inputs
itemslist of objects · required — A list of objects (rows), e.g. a data table query result or a JSON array.keyslist · required — In priority order: "amount desc", "-amount", "name asc text", or { "field", "direction": "asc" | "desc", "type": "auto" | "number" | "text" | "date" }.
Outputs
itemslist of objects — The sorted items.countinteger — How many items were returned.
Group & aggregate group
Group items by one or more fields and compute aggregates per group: count, count_distinct, sum, avg, min, max, first, last or list. Each group row has the group fields, count and the aggregates.
Inputs
itemslist of objects · required — A list of objects (rows), e.g. a data table query result or a JSON array.group_bylist of strings · required — The field(s) to group by.aggregateslist — "sum:amount", "avg(amount) as average", "count_distinct:product" or { "op": "sum", "field": "amount", "as": "total" }. The default name is op_field (sum_amount).
Outputs
groupslist of objects — One row per group, in order of first appearance: the group fields, count and each aggregate.countinteger
Pivot pivot
Turn values of one field into columns: one row per Rows key, one column per distinct Column value, aggregating Value (sum, count, avg, min, max, first, last).
Inputs
itemslist of objects · required — A list of objects (rows), e.g. a data table query result or a JSON array.rowslist of strings · required — Field(s) that identify an output row.columnstring · required — Field whose values become columns.valuestring — Field to aggregate in each cell (not needed for count).aggregatechoice · sum / count / avg / min / max / first / … — How to combine several values in one cell.filljson — Value for a row/column with no items (default empty; e.g. 0).
Outputs
itemslist of objects — One row per Rows key with a property per column.columnslist of strings — The column values, in order of first appearance.countinteger — How many items were returned.
Unpivot unpivot
Turn columns into rows: for every item, one row per column with the column name and its value, keeping identifier fields.
Inputs
itemslist of objects · required — A list of objects with one column per measure.keep_fieldslist of strings — Identifier fields copied to every row.columnslist of strings — The fields to turn into rows (default: every field not kept).name_fieldstring — Field that receives the column name.value_fieldstring — Field that receives the value.skip_emptyboolean — Leave out rows whose value is empty.
Outputs
itemslist of objects — One row per item and column.countinteger — How many items were returned.
Remove duplicates dedupe
Remove duplicate items, comparing chosen fields (or whole items, which also works for lists of plain values), keeping the first or the last of each.
Inputs
itemslist · required — A list of objects or plain values.fieldslist of strings — Fields that make an item unique (default: the whole item).keepchoice · first / last — Which duplicate to keep (the order of the list is kept).case_sensitiveboolean — Off: "A@x.io" and "a@x.io" are duplicates.
Outputs
itemslist of jsons — The unique items.countinteger — How many items were returned.removedinteger — How many duplicates were removed.
Join join
Combine two lists on matching keys (inner, left, right or full outer). Right fields whose names clash with left fields get the prefix right_ (or your own prefix on every right field).
Inputs
leftlist of objects · required — The main list of objects.rightlist of objects · required — The list to look up.left_keylist of strings · required — Key field(s) in the left list.right_keylist of strings — Key field(s) in the right list, in the same order (default: the left key names).typechoice · inner / left / right / full — Which unmatched rows to keep.right_prefixstring — Optional prefix for every field from the right list.case_sensitiveboolean — Off: keys match ignoring case.
Outputs
itemslist of objects — The joined rows (one per match).countinteger — How many items were returned.unmatched_leftinteger — Left rows with no match.unmatched_rightinteger — Right rows with no match.
CSV to JSON csv_to_json
Parse CSV text (RFC 4180: quoted fields, "" escapes, line breaks in quotes) into a list of objects, using the header row for field names and inferring numbers and true/false.
Inputs
csvtext · required — The CSV text.delimiterstring — auto (detect), comma, semicolon, tab, pipe or the character(s).headerboolean — The first row holds the field names (off: column_1, column_2, …).infer_typesboolean — Turn numbers and true/false into numbers and booleans, empty cells into empty values (numbers with leading zeros stay text).trimboolean — Trim spaces around values.
Outputs
itemslist of objects — One object per data row.columnslist of strings — The field names.countinteger — How many data rows.
JSON to CSV json_to_csv
Write a list of objects as CSV (RFC 4180 quoting, CRLF line endings by default), with chosen columns and headers. Text that a spreadsheet would run as a formula is escaped by default.
Inputs
itemslist of objects · required — A list of objects (rows), e.g. a data table query result or a JSON array.columnslist of strings — Fields to write, optionally "field as Header" (default: every field, in order of appearance).delimiterstring — Field separator (tab for tab-separated).headerboolean — Write the column names first.line_endingchoice · crlf / lf — CRLF (RFC 4180, Excel) or LF.escape_formulasboolean — Prefix text starting with = + - @ with ' so spreadsheets do not run it (CSV injection).
Outputs
csvtextcolumnslist of strings — The header names.countinteger — How many data rows.
Network
None. The action cannot make requests.
Data and files
None. It only sees its inputs.
Secrets
None.
Install
Install it in your workspace
Works on UserTasks cloud and any self-hosted instance. The action lands in Extensibility → Workflow actions, enabled, with its sample workflow.
In the app
- Open Extensibility → Workflow actions → Browse gallery and choose Install on Data transform. Or download workflow-action-data-transform-1.0.0.utpkg and drag it onto Automations → Packages.
- The wizard verifies the checksum, every file hash and the signature, and shows what it adds.
- In the workflow designer, drag Data transform from Custom actions, pick a function and fill its inputs. Save & Test shows the typed outputs.
With the CLI
curl -fLO https://usertasks.com/gallery/workflow-action-data-transform-1.0.0.utpkg
usertasks packages install workflow-action-data-transform-1.0.0.utpkg
usertasks activities test ut_data -f '{"operation":"select"}'Verify the download
- File SHA-256
- b53345e5b00f2bec410e92473940f049a177e501d56a898819cf39ea5baf55ed
- Package checksum (what is signed)
- sha256:fb15dff8f4a7c81dddb1e96479981369538feeb661be57f536eb03177b5cce1c
- Module SHA-256
- d0401d975e2f88d541f9ba45202facd27aaaf4e4f6775b0c41f14ae1c72b9119
- Signer
- UserTasks Gallery · 49:f3:04:90:47:02:4d:07:28:61:3d:dc:bb:aa:db:11
More workflow actions
All actionsCrypto & IDs
ut_crypto
UUID v4/v7 and ULIDs, SHA hashes, HMAC webhook signatures with a stored secret, base64/hex/URL encoding, random and one-time codes, and HS256 JWTs.
UUID · ULID · Hash · HMAC signature · Encode · Decode · +4 more
10 functions Uses secrets
Dates & business calendar
ut_dates
Business days around holidays, SLA due times in working hours across time zones and DST, durations and ages, recurring schedules and fiscal periods.
Add business days · SLA due time · Duration / age · Convert time zone · Next occurrences · Fiscal period
6 functions Pure computation
Math & statistics
ut_math
Safe formulas with variables, exact rounding, statistics and percentiles, compound interest, loan schedules and unit conversion in one workflow step.
Evaluate formula · Round · Statistics · Percentile · Compound interest · Loan schedule · +1 more
7 functions Pure computation