Sunday 19 September 2021

ADF Build - missing publish_config.json

Case
I'm using the new and improved ARM export via Npm to generated an ARM template for my Data Factory so I can deploy it to the next environment, but the Validate step and the Validate and Generate ARM template step both throw an error sayin that the publish_config.json file can't be found. This file isn't mentioned in the steps from the documentation. How do I add this file and what content should be in it?
Unable to read file: publish_config.json


















ERROR === LocalFileClientService: Unable to read file: D:\a\1\publish_config.json, error: {"stack":"Error: ENOENT: no such file or directory, open 'D:\\a\\1\\publish_config.json'","message":"ENOENT: no such file or directory, open 'D:\\a\\1\\publish_config.json'","errno":-4058,"code":"ENOENT","syscall":"open","path":"D:\\a\\1\\publish_config.json"}
ERROR === PublishConfigService: _getLatestPublishConfig - Unable to process publish config file, error: {"stack":"Error: ENOENT: no such file or directory, open 'D:\\a\\1\\publish_config.json'","message":"ENOENT: no such file or directory, open 'D:\\a\\1\\publish_config.json'","errno":-4058,"code":"ENOENT","syscall":"open","path":"D:\\a\\1\\publish_config.json"}
Solution
While it indeed looks like a real error it doesn't stop the DevOps pipeline. The old method of publishing ADF changes to an other ADF did create this file automatically in the adf_publish branch when you hit the Publish button the the Data Factory Gui. So probably it isn't used any more, but we still want to get rid of annoying errors!

You can solve this by manually adding the missing file:

1) Add new file to repos
To solve it go to the Azure DevOps repository and locate the folder where ADF stores the pipeline, dataset and factory files (in subfolders). Click on the +New button and create a file called publish_config.json.
Add new file to repository (in root of ADF)










2) Add JSON content
The content of the new file should be the name of your publishing branch when you configured GIT for ADF in the following JSON format:
{"publishBranch": "factory/adf_publish"}

Add the publishing branch in the following format









3) The result
Now the new file is available for the Npm task in the pipeline. Run that DevOps pipeline again and you will notice that the error message won't appear in the logs.
publish_config.json is now available for the pipeline













Conclusion
In this post you learned how to avoid the error message about the missing publish_config.json file. Not very satisfying that it is still unknown why this file was missing and if it is still used by the process. Please add comment below if you found more details.

In a next post we will describe the entire Data Factory ARM deployment where you don't need to hit that annoying Publish button within the Data Factory GUI. Everything (CI and CD) will be a YAML pipeline).

thx to colleague Roelof Jonkers for helping

1 comment:

  1. thanks, i have the same problem.. but my archive already exist.. and the problem persist.

    ReplyDelete

All comments will be verified first to avoid URL spammers. यूआरएल स्पैमर से बचने के लिए सभी टिप्पणियों को पहले सत्यापित किया जाएगा।