An Introduction to Projects in Data Pipelines

You can create 'projects' to organize you data pipelines into groups. Switch between projects by selecting one in the dropdown.

In project view, only the pipelines that belong to the selected project are visible in the pipeline list and under 'Recent Runs' in the Dashboard.

When filtering recent runs by pipeline name or job id, only the pipelines within the selected project will be in the results.

The same pipeline can be added to multiple projects. This is useful when multiple users are collaborating on the same pipeline. An analyst and an engineer might want to categorize a pipeline differently.

When you create a pipeline, it is automatically added to the currently selected project. Using projects is optional. To create a pipeline that does NOT belong to a project, select 'View All' in the project dropdown then 'Add Pipeline'.

You can always add an existing pipeline to any project. To do this, switch to the project by selecting it in the dropdown, then click 'Add Pipeline' and add or remove existing pipelines from the list.

Projects are visible to anyone within your Data Pipelines organization, so any project you create will be accessible by your colleagues within the same organisation.

To delete a project, click the trashcan icon while the project is selected. Deleting a project does not delete the pipelines it contains so you can safely delete a project without first removing pipelines from it.

There is no limit on the number of projects you can create.