Publish ADF via DevOps

In this series of posts you will find detailed steps on how to deploy your development Azure Data Factory to the rest of the DTAP environments via Azure DevOps. Besides the build and publish process we also show you how to adjust the global parameters, linked services and triggers during the deployment to fit the requirements of each specific environment. The focus in on YAML pipelines, but you can ofcourse also still use the classic editor without YAML.





All our ADF DevOps posts:

A post not specific to Azure Data Factory, but a quick introduction to Azure DevOps for developers describing each part of the service.
The first main post in the series is about creating the build part of the DevOps pipeline where the ARM template is generated that will be used for publising ADF to the next environments. The node.js solution allows you to stop using that cursed publish button that you always forget to click.
This post shows you how to solve the missing publish_config.json error when you want to build the ARM template. It seems to be a relic of the past when we were forced to click the publish button, but it is very easy to get rid of by manually creating the file.
This post describes how to solve the missing arm-template-parameter-definition.json error when you want to build the ARM template. It is easy to solve and it would probably solve itself when you start overriding parameters, global variables and triggers.
This second main post is all about publishing the previously generated ARM template for ADF to various DTAP environments. The best tip here is to use incremtal deployment to avoid those stunned faces when seeing your wiped out resource group. Next step is overriding those parameters and linked services.
This follow-up post of the deployment post shows you how to override the global parameter values during deployment via DevOps. This allows you to use different parameter values within your DTAP environments.
The next follow-up post of the deployment post shows you how to change linked service properties during deployment via DevOps. This allows you to for example use a different Azure Key Vault for development and production environments.
The last follow-up post of the deployment post has been split into a PowerShell post and a nocode post to disable or enable certain triggers for specific environment.  You probably don't want to let all your DTAP enviroments of ADF simultaneously query your sources.
The pre- and post-deployment script from Microsoft could return a ResourceGroupNotFound error when your Service Principal has access to multiple Azure subscriptions. This post shows you hove to solve that by adding just a few lines of code.
This post not only works for ADF deployments in ADF. It shows you where all your files are on the DevOps agent. This is especially handy when you're still working on your DevOps pipeline and you need to set paths for the various tasks. Besides the very handy treeview it also show how to see the content of files.
All ADF DevOps posts in chronological order:

    No comments:

    Post a Comment

    All comments will be verified first to avoid URL spammers. यूआरएल स्पैमर से बचने के लिए सभी टिप्पणियों को पहले सत्यापित किया जाएगा।