Running & Monitoring Pipelines
Run a pipeline, follow its progress and read its logs.
A saved pipeline runs on demand. Each run keeps its own status, action results and logs, and stays in the Runs list.
Starting a Run
Select Run on the pipeline's row in the Pipelines list, or Run in the top bar of the pipeline builder. A pipeline has at most one run in flight at a time, and a paused run counts as in flight.
The Runs Page
The Runs page in the left menu lists runs, each with the pipeline name, its status, when it ran, how long it took and how many records it handled. Two filters narrow the list: a switch for "Only runs that have not finished", and a status dropdown. Select a run to open it.
A Run's Page
A run's page shows:
- The pipeline name, the run's status, and View pipeline.
- Progress: one bar for the whole run, with the percentage and the status word.
- Details: when it started and completed, and the number of records.
- Actions: each action of the pipeline with its own status, and a record count where the action reports one.
- Logs: the run's log entries with the number of warnings and errors.
While a run is in flight, the page also has a Cancel button.
Logs
The log table has the columns Time, Level, Logger, Environment and Message. Filter by level (the default shows information and above), by logger and by a time range, change the order, Follow the log as new entries arrive, or Export it. Warnings show in amber.
Run Statuses
A run moves through these statuses. The Runs page filter lists them all.
| Status | Meaning |
|---|---|
| Pending | The run has been created and is waiting to be prepared. |
| Pending approval | The run needs an approval before it can start. |
| Preparing | The agents are analysing the entities to move. |
| Ready | Preparation is done and the run is about to start. |
| Executing, Extracting, Transferring, Loading | The run is working. Which of these you see depends on the actions in the pipeline. |
| Completing | The run is finishing. |
| Paused | The run stopped in front of a step because the workspace's remaining record allowance could not cover it. Nothing failed, and the work that already ran stays applied. |
| Completed | The run finished. |
| Failed | The run ended with an error. |
| Cancelled | The run was cancelled. |
| Rejected | The run was refused because it would break a quota or plan limit. |
A paused run
A paused run waits until you decide. Check the environment, then select Resume run to try the next step again against your current allowance. The environment may have changed since the run paused.
Cancelling a Run
- Open the run and select Cancel.
- Read the dialog and confirm, or select Keep Running.
Cancelling stops the run from starting any further work and asks the agents to finalise. Work already under way is not interrupted: it finishes, and anything it has written stays written. A step that changes data in place can therefore complete in full even though the run ends as Cancelled.
Troubleshooting
A run fails or is rejected
Open the run and read its Logs, filtering to errors. The log names the environment each entry came from.
A run is paused
Your workspace's record allowance could not cover the next step. The run's page shows the reason. Compare your plan's limits with the amount of data, then resume the run once the allowance covers it.