Creating a Pipeline
Build a data pipeline to copy or transform data between environments.
A pipeline defines a repeatable data operation between two environments. It specifies the source and target environments, which entities to include, and any transformations to apply during transfer. Once created, a pipeline can be executed on demand as many times as needed.
Quick Create vs Custom
DMM Infinity offers two ways to create a pipeline:
| Method | Best For | Description |
|---|---|---|
| Quick Create | Simple copy operations | Select a source, target, and entities â DMM Infinity configures everything else with sensible defaults. No transformations are applied. |
| Custom Pipeline | Complex operations with transformations | Full access to the pipeline builder with step-by-step configuration, including entity selection, field-level transformations, and advanced options. |
Start with Quick Create for your first pipeline. You can always edit it later to add transformations or adjust settings.
Pipeline Builder Walkthrough
The custom pipeline builder guides you through five steps:
Choose Action Type
Select the type of operation this pipeline performs:
- Copy â transfer data from source to target, creating or updating records in the target environment
- Delete â remove records from the target environment that match specified criteria
- Transform â copy data while applying field-level transformations (anonymisation, masking, scrambling)
The Transform action type includes all Copy functionality plus transformation capabilities. If you plan to anonymise any fields, choose Transform.
Select Environments
Choose the source and target environments from your connected environments:
- Source â the environment data is read from
- Target â the environment data is written to
Both environments must be on the same platform. For example, you cannot create a pipeline between a Mendix source and an OutSystems target.
Select Entities
Choose which entities (database tables) to include in the pipeline. The builder displays all available entities from the source environment with their field counts.
- Use the search bar to filter entities by name.
- Check the box next to each entity you want to include.
- The builder automatically resolves entity dependencies â if Entity A references Entity B, both are included.
- Review the total record count estimate to understand the scope of the operation.
Configure Transformations (Optional)
If you selected the Transform action type, you can now configure field-level transformations:
- Expand each entity to see its fields.
- For each field, select a transformation type from the dropdown: Hide, Static Value, Anonymize, Scramble, Regexp, or Ignore.
- Configure transformation parameters (e.g., the static value to use, or the regex pattern).
- Fields without a transformation are copied as-is.
See the Transformations Reference for detailed documentation on each transformation type.
Review and Save
Review the pipeline configuration summary:
- Action type and direction (source â target)
- Number of entities and estimated record count
- Applied transformations (if any)
- Give the pipeline a descriptive name (e.g., "ACC â DEV Data Refresh" or "PRD â ACC Anonymised Copy").
- Click Save to create the pipeline.
Editing a Pipeline
You can edit any pipeline that is not currently running:
- Navigate to the Pipelines page.
- Click the pipeline you want to edit.
- Click the Edit button to re-enter the pipeline builder.
- Make your changes â you can modify entities, transformations, environments, or the pipeline name.
- Click Save to apply the changes.
Changing the source or target environment resets entity selections and transformations, since the available entities may differ between environments.
Deleting a Pipeline
To permanently remove a pipeline:
- Navigate to the Pipelines page.
- Click the pipeline you want to delete.
- Click the Delete button and confirm the action.
Deleting a pipeline does not affect any data that was previously transferred. It only removes the pipeline configuration and its execution history from DMM Infinity.