Monday, 18 February 2019

ADF snack: Pipeline templates

Case
I'm regularly creating the same kind of pipelines in Azure Data Factory. Is there a way to copy or reuse them without writing JSON?
ADF Templates













Solution
If you are working within one Data Factory then you could just clone the pipeline by clicking the three dots behind the template that you want to copy. After that, a new copy of the pipeline will be created.
Clone a pipeline





















But Microsoft has also recently introduced pipeline templates which could help you with this. The template gallery contains several templates from Microsoft, but you can also add your own templates. However using a Git Repository is required for this option. To add a new pipeline from a template click on the + icon after the filter textbox and then choose Pipeline from template. Now find the right template and fill in all the required fields for each activity.
Choosing a pipeline template














To add your own pipeline as a template you first need to open your pipeline. Then press the Save as template option in the top menu. Now you can give your new template a name and then press the Save template button in the bottom right.
Creating a new pipeline template














Summary
In this post we showed you how to use the new pipeline template option in Azure Data Factory. This could be handy when you are creating a lot of similar pipelines, but the templates from Microsoft could also be a quick start when you don't have any templates and you are starting from scratch.







Related Posts Plugin for WordPress, Blogger...