Flexible Azure Service Bus Resource Manager (ARM) Templates

Bending ARM templates to your will with a sprinkling of parameters and PowerShell.

I’ve recently had cause to be deploying quite a number of Azure resources with Azure Resource Manager (ARM) templates and PowerShell. In most cases the resources could be created with a relatively simple template and a small number of parameters for configurability to cater for different environments (i.e. Dev, Test, etc) and customers.

The one place where this became more complex was the Azure Service Bus deployments. This was due the the nesting of Namespaces, Topics, Subscriptions and Rules and the potential variability in the number of topics and subscriptions required for an environment or customer.

I’m going to dissect a template that, along with some PowerShell, will allow an arbitrary number of topics and subscriptions to be defined for a namespace, where each subscription can have a single rule (i.e. the $Default rule).

ARM templates allow you to deploy multiple instances of a resource definition through the rather obtuse “copy”

copy“: {
name“: “topicCopy“,
count“: “[length(parameters(‘topicList’))]
}

In this example, “topicCopy” is an identifier for the group of resource instances being deployed and count“: “[length(parameters(‘topicList’))] sets the number of copies that will be made, in this case base on the size of an array parameter provided to the template containing a list of topic names to be create on the service bus. This feature is what allows us to create an arbitrary number of topics and subscriptions.

The first parameter simply defines the service bus Namespace to be created by the template. You’ll note in the full template (at the end) there is an additional Service Bus API Version parameter with a default value. See the Azure Service Bus documentation for a further discussion of this if needed.

serviceBusNamespaceName“: {
type“: “string“,
metadata“: {
description“: “Name of the Service Bus namespace
}
}

The next parameter, topicList, defines the list of Topics that will be created (or updated) on the service bus Namespace defined above. This list can contain any number of topics.

topicList“: {
type“: “array“,
metadata“: {
description“: “a list of topics that will be created
}
}

The subscriptionList parameter is where it starts getting a little tricky. This is an array of Subscriptions to be created for the Topics defined above, however, the naming of the subscriptions must be specifically provided as “topicName/subscriptionName”. For example, if you the topic list above had a topic called “cashMoney” and you wanted to create a subscription on that topic called “inTheBank”, the value in this parameter array must be “cashMoney/inTheBank”. If you don’t want to create a subscription on a given topic, simply do not provide one in this list.

subscriptionList“: {
type“: “array“,
metadata“: {
description“: “a list of subscritpions to be deployed – these must be named for the topics provided above as topic/subscription
}
},

The subscriptionFilterList then contains the sqlFilter text for the $Default filter. The values in this array will be assigned to the subscriptions above based on the corresponding index positions in the arrays. For this reason, filter conditions are not optional and if you do not want filter messages for a given subscription, you MUST provide the default “1=1” condition for the filter.

subscriptionFilterList“: {
type“: “array“,
metadata“: {
description“: “a list of filter rules (e.g. CountryCode=’XX’) that must correspond to the subscriptions list above – these are correlated by index only
}
}

That covers off the parameters that must be supplied to the ARM template to make it create any number of topics and subscriptions on a service bus namespace. You could provide these parameters in a parameters JSON file and pass this to the New-AzureRmResourceGroupDeployment command, but that leaves you with a parameter file per environment and client.

Instead, you could consider the following PowerShell. You define your Topics, Subscriptions and Filters in a Hashtable of Hashtable’s. For example:

$serviceBusTopicsSubsFilters = @{
"topicOne" = @{
"SubscriptionOne" = "1=1";
};
"topicTwo" = @{
"SubsubscriptionOne" = "SomeProperty=1";
"SubsubscriptionTwo" = "SomeProperty=2";
"SubsubscriptionThree" = "SomeProperty=3";
};
"topicThree" = @{
"SubscriptionOne" = "AnotherProperty='SomeText'";
};
}

These hashmaps can then be processed into the necessary arrays to be passed in as parameters to the ARM template using the following

$serviceBusTopics = [System.Collections.ArrayList]@()
$serviceBusSubs = [System.Collections.ArrayList]@()
$serviceBusFilters = [System.Collections.ArrayList]@()

$serviceBusTopicsSubsFilters.Keys | % {
$currTopic = $_
$serviceBusTopics.Add($currTopic) | Out-Null
$serviceBusTopicsSubsFilters.Item($currTopic).Keys | % {
$currSub = $_
$serviceBusSubs.Add("$currTopic/$currSub") | Out-Null
$serviceBusFilters.Add($serviceBusTopicsSubsFilters[$currTopic][$currSub]) | Out-Null
}
}

You can then perform the resource manager deployment with the following command

New-AzureRmResourceGroupDeployment `
-ResourceGroupName $ResourceGroupName `
-TemplateFile $TemplateFile `
-serviceBusNamespaceName $Namespace `
-topicList $serviceBusTopics `
-subscriptionList $serviceBusSubs `
-subscriptionFilterList $serviceBusFilters

If you were not already aware of this, PowerShell is clever enough to pass through the command line parameters provided into the parameters of the ARM template. The last four command line parameters in the example above are actually the four template parameters discussed earlier.

I feel like ARM templates have a lot of catching up to do with AWS CloudFormation scripts when it comes to conditional logic and functions in the templates, but in this case I was able to reach my goal with solution that was not overly complex.

With a little further modification, you could extend this to allow you to set all the properties of a topic, subscription and potentially add other rules. This will clearly increase the complexity, but if that’s what you need then go for it. If you were to start adding support for queues, relays, etc. I imagine that you would reach the point of diminishing returns quite quickly.

I would accept the argument that the creation of the Hashtables only to convert them to arrays is wasteful, but to my mind it lowers the typo error rate in the case where many subscriptions are mapped to many topics in the subscriptions list array. You could also argue that this offers little benefit over providing these parameters through a parameter JSON file again, I would point to the typo error rate.

In my context, the service bus was one of many resources that was being deployed. Being able to hoist the parameters to the top of the master deployment PowerShell script meant that every option and parameter was collected into a single location and was readily visible.

I do hope this proves helpful to somebody else.

 

Full Template

{
$schema“: “https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#“,
contentVersion“: “1.0.0.0“,
parameters“: {
serviceBusNamespaceName“: {
type“: “string“,
metadata“: {
description“: “Name of the Service Bus namespace
}
},
serviceBusApiVersion“: {
type“: “string“,
defaultValue“: “2015-08-01“,
metadata“: {
description“: “Service Bus ApiVersion used by the template
}
},
topicList“: {
type“: “array“,
metadata“: {
description“: “a list of topics that will be created
}
},
subscriptionList“: {
type“: “array“,
metadata“: {
description“: “a list of subscritpions to be deployed – these must be named for the topics provided above as topic/subscription
}
},
subscriptionFilterList“: {
type“: “array“,
metadata“: {
description“: “a list of filter rules (e.g. CountryCode=’XX’) that must correspond to the subscriptions list above – these are correlated by index only
}
}
},
variables“: {
location“: “[resourceGroup().location]“,
sbVersion“: “[parameters(‘serviceBusApiVersion’)]“,
defaultSASKeyName“: “RootManageSharedAccessKey“,
authRuleResourceId“: “[resourceId(‘Microsoft.ServiceBus/namespaces/authorizationRules’, parameters(‘serviceBusNamespaceName’), variables(‘defaultSASKeyName’))]
},
resources“: [
{
apiVersion“: “[variables(‘sbVersion’)]“,
name“: “[parameters(‘serviceBusNamespaceName’)]“,
type“: “Microsoft.ServiceBus/Namespaces“,
location“: “[variables(‘location’)]“,
sku“: {
name“: “Standard“,
tier“: “Standard
}
},
{
apiVersion“: “[variables(‘sbVersion’)]“,
name“: “[concat(parameters(‘serviceBusNamespaceName’), ‘/’, parameters(‘topicList’)[copyIndex()])]“,
type“: “Microsoft.ServiceBus/Namespaces/Topics“,
copy“: {
name“: “topicCopy“,
count“: “[length(parameters(‘topicList’))]
},
dependsOn“: [
[concat(‘Microsoft.ServiceBus/namespaces/’, parameters(‘serviceBusNamespaceName’))]
],
properties“: {
defaultMessageTimeToLive“: “01:00:00“,
maxSizeInMegabytes“: 1024,
requiresDuplicateDetection“: false,
enableBatchedOperations“: true,

filteringMessagesBeforePublishing“: false,
isAnonymousAccessible“: false,
supportOrdering“: true,
autoDeleteOnIdle“: “14.00:00:00“,
enablePartitioning“: false,
enableExpress“: false
}
},
{
apiVersion“: “[variables(‘sbVersion’)]“,
name“: “[concat(parameters(‘serviceBusNamespaceName’), ‘/’, parameters(‘subscriptionList’)[copyIndex()])]“,
type“: “Microsoft.ServiceBus/Namespaces/Topics/Subscriptions“,
copy“: {
name“: “subsDeployment“,
count“: “[length(parameters(‘subscriptionList’))]
},
dependsOn“: [
topicCopy
],
properties“: {
lockDuration“: “00:01:00“,
requiresSession“: false,
defaultMessageTimeToLive“: “01:00:00“,
deadLetteringOnMessageExpiration“: false,
deadLetteringOnFilterEvaluationExceptions“: true,
maxDeliveryCount“: 10,
enableBatchedOperations“: true,
autoDeleteOnIdle“: “14.00:00:00
},
resources“: [
{
apiVersion“: “[variables(‘sbVersion’)]“,
name“: “$Default“,
type“: “Rules“,
dependsOn“: [
[split(parameters(‘subscriptionList’)[copyIndex()], ‘/’)[1]]
],
properties“: {
filter“: {
sqlExpression“: “[parameters(‘subscriptionFilterList’)[copyIndex()]]
}
}
}
]
}
],
outputs“: {
NamespaceConnectionString“: {
type“: “string“,
value“: “[listkeys(variables(‘authRuleResourceId’), variables(‘sbVersion’)).primaryConnectionString]
},
SharedAccessPolicyPrimaryKey“: {
type“: “string“,
value“: “[listkeys(variables(‘authRuleResourceId’), variables(‘sbVersion’)).primaryKey]
}
}
}

Categories

Recent Posts

Tags

Kurt Written by: