writing-mappings

Writing Mappings and Transforms

Mappings and transforms are the data reshaping layer in Celigo integrations. They control how fields from one system translate into fields for another. Mappings are used across flows, APIs, and tools.

Mapping Systems

Four systems handle data reshaping:

• Mapper 2.0 -- modern recursive field mapping on imports (mappings[] array). Handles nested objects, arrays of any depth, lookups, conditionals, and date conversions. Default for new imports on all adaptor types except NetSuite and Salesforce
• Mapper 1.0 -- legacy flat mapping on NetSuite and Salesforce imports (mapping.fields[] / mapping.lists[]). Body-level and sublist fields in separate flat arrays. Also present on many older HTTP/FTP/RDBMS imports created before Mapper 2.0 existed
• Transformation 2.0 -- rule-based data reshaping on exports (transform.expression.rulesTwoDotZero). Uses the same Mapper 2.0 schema internally. Two modes: "create" (build new record from scratch) or "modify" (edit fields on existing record, unmapped fields pass through)
• Response mapping -- simple extract/generate pairs that carry data from a lookup or import response back into the record (responseMapping on flow pageProcessors[]). Uses Transformation 1.0 syntax

Lookups are shared across all systems -- static key-value maps or references to LookupCache resources for large/dynamic datasets. NetSuite imports use a distinct lookup system that queries live NetSuite records.

Direction decides the tool. Mappings translate data going out to a destination -- every import needs them, because the in-flight record almost never matches what the destination expects. Transformations reshape data coming in -- on exports, listeners, and API/tool entry stages. Never use an upstream transformation to match a destination's shape; that's the destination import's mapping. Transformations earn their keep in two situations: multiple sources feeding one pipeline (reshape each new source to the canonical record shape the existing steps expect) and genuinely messy source data (flatten deep nesting once at entry instead of fighting it in every downstream mapping). With a single well-shaped source, don't add a transform just because you can -- and skip identity transforms that rename nothing.

Quick Reference

Which Mapping System?

Editing existing imports: Many existing imports use Mapper 1.0 even for HTTP and RDBMS adaptor types (pre-dating Mapper 2.0). Always check whether an import uses mappings[] (2.0) or mapping.fields[] (1.0) before modifying -- never mix the two.

Schema Index

Related Skills

• configuring-imports > Quick Reference -- import adaptor types, operation logic, hooks
• configuring-exports > Quick Reference -- export adaptor types, delta syncs, webhooks
• writing-handlebars > Quick Reference -- Handlebars expressions used inside mapping extract fields
• building-flows > How to Build a Flow -- wiring exports and imports into a flow pipeline

Mapper 2.0 Workflow

The mappings[] array is recursive -- a mapping can contain nested child mappings of the same structure to any depth. This is the core design principle.

1. Check the existing resource

Before modifying mappings, always retrieve the current state of the resource. Check whether it uses Mapper 2.0 (mappings[]) or Mapper 1.0 (mapping.fields[]).

celigo imports get <importId>
celigo account search <keyword>

2. Understand the source data shape

Invoke the upstream export to see real records, or query the source system's metadata for the full field list.

celigo

3. Understand the target data shape

Query metadata for the target system to discover required fields and types.

4. Choose the input context

The input context controls what data is available to extract paths. Set via the "Input context" dropdown in the mapper UI:

• record (default) -- extract paths reference the record directly. $.user_id accesses the user_id field on the record
• envelope -- extract paths reference a wrapper object containing record, job, settings (with connection, flow, integration, flowGrouping), iClient, and import. Record fields shift to $.record.user_id, but you gain access to metadata like $.settings.connection.api_username, $.job.type, $.settings.flow.fieldName

When to use envelope: APIs and tools where you need request context (headers, path params, query params, connection settings) directly in mappings without Handlebars. Also useful on transforms at the beginning of API/tool steps where the envelope exposes the full request context. Envelope context eliminates the need for {{settings.connection.fieldName}} Handlebars expressions -- use $.settings.connection.fieldName instead.

5. Map by data type

Every mapping needs three properties: generate (target field name), dataType (output type), and extract (how to get data from source).

Extract supports three patterns (distinguished by syntax):

• JSON path -- starts with $. (e.g., $.customer.email). Always references the top-level root, even in nested mappings
• Handlebars -- contains {{ (e.g., {{record.firstName}} {{record.lastName}}). For computed values
• Hard-coded -- plain string literal (e.g., "Active", "USD"). Neither $. prefix nor {{

Simple types (string, number, boolean, date) -- direct field-to-field mapping. For dates, set extractDateFormat/generateDateFormat for conversion.

Objects -- set dataType: "object", add child mappings in the mappings[] array. Never use dot notation in generate.

Arrays -- set dataType to an array type (stringarray, numberarray, booleanarray, objectarray, arrayarray) and configure buildArrayHelper[]. Three patterns for object arrays:

• Extract only -- pull existing objects from source (extract: "$.items[*]")
• Mappings only -- construct objects from individual fields (each buildArrayHelper entry creates one array element)
• Extract + mappings -- iterate a source array and reshape each element. Uses the composite object mechanism: array brackets [*] in the extract path collapse to single objects inside the mappings, so $.orders[*].items[*] becomes $.orders.items.fieldName in child extract paths. Parent context remains accessible (e.g., $.orders.id, $.customerName)

6. Add lookups for value translation

Define lookups alongside mappings and reference them by name via lookupName on any mapping.

• Static -- map object with key-value pairs. Best for small, fixed sets (country codes, status values)
• Dynamic -- _lookupCacheId referencing a LookupCache resource, with optional extract JSON path to pull a specific field from the cached object
• Set allowFailures: true + default to continue processing when lookup keys are missing

7. Add conditionals where needed

Control when a mapping applies: record_created (only on insert), record_updated (only on update), or extract_not_empty (skip when source is null/empty).

Schema reference

All Mapper 2.0 field definitions: mappings.yml, lookups.yml

Transformation 2.0 Workflow

Transformation 2.0 reshapes data on exports before it enters the pipeline. It wraps Mapper 2.0 syntax in a transform envelope with a mode selector.

1. Check the existing resource

Before modifying transforms, retrieve the current export to inspect any existing transform configuration.

2. Choose the mode

• create -- build a completely new record. Only mapped fields appear in output. Use when the output structure differs significantly from the source
• modify -- edit specific fields on the existing record. Unmapped fields pass through unchanged. Use for surgical adjustments (rename, add, remove a few fields)

3. Write the mappings

Same Mapper 2.0 syntax: generate, dataType, extract, nested mappings, buildArrayHelper, lookups. Everything described in the Mapper 2.0 section above applies here, including input context.

Input context is especially valuable on transforms for APIs and tools -- set it to envelope to access the full request context (headers, path params, query params, connection settings) directly via JSON path instead of Handlebars.

4. Configure the transform envelope

Set transform.type: "expression", expression.version: "2", then place mappings[] and lookups[] under expression.rulesTwoDotZero with the chosen mode.

Script alternative: Set transform.type: "script" with script._scriptId and script.function for programmatic transforms when expression rules aren't sufficient.

Schema reference

All Transformation 2.0 field definitions: transform.yml, mappings.yml, lookups.yml

Mapper 1.0 Reference (NetSuite and Salesforce)

NetSuite and Salesforce imports use the older flat mapping structure. Two arrays within the mapping object:

• mapping.fields[] -- body-level field mappings. Each entry has extract (source path) or hardCodedValue (static value), generate (target field ID), and optional lookupName, dataType, internalId, immutable, discardIfEmpty, conditional
• mapping.lists[] -- sublist/line-item mappings. Each entry has generate (sublist ID, e.g., "item"), jsonPath (source array path), and fields[] (column mappings with the same properties as body fields, plus isKey for matching existing lines)

NetSuite lookups are different -- they query live NetSuite records using recordType, searchField, resultField, and operator. Not static maps. Defined in netsuite_da.lookups[], referenced by lookupName in field mappings. To discover valid field IDs for searchField and resultField, run celigo metadata fields <connectionId> <recordType> — the returned field IDs are the exact values to use.

Salesforce lookups follow the same Mapper 1.0 pattern but the lookup structure is simpler.

Sublist field discovery: For NetSuite mapping.lists[].generate, the sublist name (e.g., "item", "addressbook") comes from celigo metadata fields <connectionId> <recordType> — sublists appear as field groups. For Salesforce related lists, use celigo metadata fields <connectionId> <sObjectType> to discover relationship fields and child object names for distributed.relatedLists[].sObjectType.

Schema reference

NetSuite Mapper 1.0: see mapping and lookups in the configuring-imports skill's netsuitedistributed.yml Salesforce Mapper 1.0: see mapping in the configuring-imports skill's salesforce.yml

Response Mapping Reference (Transformation 1.0)

Response mapping extracts fields from a lookup or import API response back into the original record. It lives on the flow's pageProcessors[] entry, not on the resource itself -- but it's planned when building the resource.

Two sections, both plain arrays (never wrap them in an object like "fields": {"type": [...]} -- the API accepts that shape but silently erases the step's mappings):

• fields[] -- field-level extract/generate pairs using dot notation
• lists[] -- array mappings with generate (target array name) and fields[] (extract/generate pairs applied per array item)

For lookup exports: the response envelope is {"statusCode": 200, "data": [...result records], "errors": []}, and every extract starts from data.

• Single result -- data[0].fieldName (or the equivalent data.0.fieldName).
• Multiple results, keep the whole array -- extract: "data", generate: "<arrayField>". The record gains an array of the result objects; downstream steps read it with normal JSON array paths (e.g. $.<arrayField>[*].fieldName in Mapper 2.0) or fan out over it with one-to-many (pathToMany: "<arrayField>").
• Multiple results, reshaped -- a lists entry whose field extracts use data[*].fieldName. Produces one array item per result; the [*] marks the array to iterate:

The [*] wildcard is only honored inside lists[].fields[].extract. In a top-level fields[].extract, data[*].fieldName (and bare data[*]) is silently ignored -- the field is never merged onto the record. Bare result-field names (fieldName without the data prefix) resolve to nothing in either section.

For imports: the response is available via _json. Use _json.fieldName (e.g., _json.id for a created record's ID, _json.output.1.content.0.text for AI model responses).

Schema reference

All response mapping field definitions: response-mapping.yml

CLI Commands

Pre-Submit Checklist

Before submitting any mapping configuration, verify:

• Correct system identified -- mappings[] for Mapper 2.0, mapping.fields[] for Mapper 1.0; never mix
• status: "Active" on every mapping entry -- API rejects entries without it
• No dot notation in generate -- use nested dataType: "object" with child mappings[] instead
• Extract paths start from root -- $. paths always reference the top-level input, even in nested mappings
• Array mappings have buildArrayHelper -- required for all *array dataTypes
• Lookups defined and referenced -- every lookupName on a mapping has a corresponding entry in lookups[]
• Date formats specified -- extractDateFormat/generateDateFormat set when dataType: "date"
• Full resource PUT -- GET the complete resource first, modify only the mapping section, PUT the whole object back
• Response mapping on the flow -- responseMapping lives on flow.pageProcessors[], not on the import resource itself
• Composite object paths adjusted -- when buildArrayHelper has both extract and mappings, child extract paths drop the [*] brackets

Gotchas

1. Existing imports may use Mapper 1.0 even for HTTP/RDBMS/FTP. Mapper 2.0 is the default for new imports, but many older imports across all adaptor types use Mapper 1.0. Always check the existing format before editing -- mappings[] means 2.0, mapping.fields[] means 1.0. Never mix.
2. Extract paths always reference the root of the input context. Even in deeply nested Mapper 2.0 mappings, $. paths start from the top level (the record in record context, or the envelope in envelope context), not the current nesting level.
3. Composite object collapses arrays to single objects. When buildArrayHelper has both extract and mappings, array brackets in the extract path are replaced with single objects in child mapping contexts. $.orders[*].items[*] becomes $.orders.items.fieldName inside the mappings.
4. Response mapping uses Transformation 1.0 syntax, not 2.0. Don't use rulesTwoDotZero structure in responseMapping. It uses simple extract/generate pairs with dot notation.
5. NetSuite lookups query live data. A NetSuite import's netsuite_da.lookups[] searches NetSuite records at runtime (recordType, searchField, resultField), unlike Mapper 2.0 static map lookups.
6. generate must not use dot notation in Mapper 2.0. Build nested structures with dataType: "object" and child mappings[]. "generate": "customer.name" silently creates a field literally named "customer.name".
7. Empty generate indicates inner array in arrayarray. For nested array structures, inner array mappings have no generate field -- this is expected, not an error.
8. Transformation 2.0 "modify" passes through unmapped fields. "Create" mode only outputs explicitly mapped fields. Choose based on whether you want a clean slate or surgical edits.
9. PUT erases omitted fields on the parent resource. When updating mappings on an import or transform on an export, always GET the full resource first, modify the mapping/transform section, then PUT the complete object. The set command handles this.
10. [*] in response mapping only works inside lists. data[*].fieldName in a top-level fields[].extract is silently ignored -- nothing is merged. For multiple lookup results use extract: "data" (whole array) or a lists entry with data[*].fieldName extracts.

Common Errors