What is an Azure Network Security Group?

First off, let’s cover what an NSG really is. They are a type of firewall for layer 3 and 4 traffic. The layer 3 and 4 is referring to the Open Systems Interconnection model. Layer 3, the Network Layer, is for Internet Protocol “IP” addresses such as 192.168.0.1. Layer 4, the Transport Layer, is for protocols like TCP, UDP, ICMP, and others.

An NSG provides firewall functionality on these two layers. Each rule can work with a single IP address or a range of IP addresses. You can also specify ports and even Azure Service Tags. Each rule has a priority and is processed in order. If one rule allows some traffic but another rule of higher priority denies the traffic then traffic is denied because of that priority. There are cases where one rule can allow a range while another rule denies a specific IP address. The traffic allowed or denied is still based on the priority and not specificity.

How does an NSG compare to Azure Firewall?

The NSG does process rules like the Azure Firewall, however there are three drastic differences. First, the Azure Firewall requires a Public IP address or a Public IP Prefix (IP range). The public IP endpoint(s) are used for the incoming and outgoing traffic. The NSG does not have the same kind of entrypoints. More on this later in this post. Second, Azure Firewall has collections and groups of rules each with their own associated priorities and capabilities such as Destination Network Address Translation “DNAT”. The NSG only has a single list of rules. Lastly, the Azure Firewall helps route traffic to its destination. The NSG does nothing in regards to routing.

NSG and Subnet Association

The NSG can have several rules but none of them are in effect until the NSG is associated to either a subnet of a Virtual Network “VNet” and/or directly to a Network Interface Card “NIC”. This association can be confusing so I want to spell this out as clearly as I can. Let’s start with a scenario of having a single Virtual Machine “VM”. You notice that in order to create the VM a NIC must be created and in the process the NIC must be associated with a subnet in a VNet. This association to the subnet it what gives the NIC its IP address.

When an NSG is associated with the subnet, the rules then apply to all traffic in that subnet. Yes, you can associate an NSG to some or all of the subnets within an VNet and also to include other subnets of other VNets. For now, I’m keeping it simple for this explanation.

In an NSG, rules are either inbound or outbound and have a Source and a Destination. If a rule is inbound and has a destination IP of that of an associated subnet, then the inbound network traffic is subjected to this rule. Consider this rule in our example.

1
2
3
4
5
6
7
Priority = 1001
Direction = Inbound
Source = Any
Source Port = Any
Destination = 192.168.0.5
Destination Port = 22
Action = Allow

This rule is stating that traffic coming from Any source that is destined to 192.168.0.5 on port 22 will be allowed. Now let’s add another rule. This time the rule is to deny all traffic bound to the same destination regardless of destination port.

1
2
3
4
5
6
7
Priority = 1000
Direction = Inbound
Source = Any
Source Port = Any
Destination = 192.168.0.5
Destination Port = Any
Action = Deny

The problem with this is that no traffic is getting to any port on that IP address. This is because the priority is set to a higher priority than the other rule allowing traffic to port 22. Priority 1000 is higher than 1001.

Note: NSG rules have a priority from 100 to 4096. The lower the number, the higher the priority!

So, for traffic to successfully reach 192.168.0.5 on port 22 we need to swap the priority.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Priority = 1000
Direction = Inbound
Source = Any
Source Port = Any
Destination = 192.168.0.5
Destination Port = 22
Action = Allow

Priority = 1001
Direction = Inbound
Source = Any
Source Port = Any
Destination = 192.168.0.5
Destination Port = Any
Action = Deny

Remember, the rules regarding the destination address of 192.168.0.5 are only applicable if the NSG is associated to the subnet range that includes that address, OR is directly associated to the NIC that has that address.

Direction of Rules

When creating a rule we need to understand where the traffic is originating and where it is destined. For example, if traffic is coming from the internet to a public IP address then the rule will be Inbound likely with a Service Tag of “Internet”. This Inbound rule might have a source of a collection of public IP addresses that are from business partners or specific people. Perhaps the traffic originated with a private IP address.

Let’s go through a quick example of a VM with a public IP address. You associate the NSG directly to the NIC of the VM. A quick note; this not advised and I explain further down. The expectation is to allow yourself direct access from your home. Since the traffic will originate from your home we need to create an Inbound rule. For this example, let’s assume that your home IP address is 1.2.3.4 and the IP address of the VMs public IP is 5.6.7.8. Create the rule as shown below and test.

1
2
3
4
5
6
7
Priority = 2000
Direction = Inbound
Source = 1.2.3.4
Source Port = Any
Destination = 5.6.7.8
Destination Port = 22
Action = Allow

Did you notice it didn’t work? Most of the details are correct except for one. This is a big item to remember. Public IP address on a NIC are first converted to their private IP address before an NSG rule is applied. So, we need to change the rule to have the destination of the private IP address.

1
2
3
4
5
6
7
Priority = 2000
Direction = Inbound
Source = 1.2.3.4
Source Port = Any
Destination = 192.168.0.5
Destination Port = 22
Action = Allow

What if the traffic is originating from code running on the VM and needs to get to a public endpoint in the internet? For this, we’ll create an Outbound rule. Why outbound? Because the traffic originates from inside the VNet it’s direction is outbound. It doesn’t make sense to have an Outbound rule with a Source of Internet so that rule will not be effective.

An Outbound rule is created with a source IP of the NIC. The destination is to some random public IP for this example. Notice the destination port is 443 but the source port is Any. When web-based traffic occurs, it first starts with an agreement to further the conversation on an open source port. It’s rare that you will set a source port. However, there are those special cases when there is tight control over that traffic. Web-based traffic is of the few types of conversations that leverage other open ports.

1
2
3
4
5
6
7
Priority = 2000
Direction = Outbound
Source = 192.168.0.5
Source Port = Any
Destination = 15.16.17.18
Destination Port = 443
Action = Allow

Wait! Don’t we already have a rule with that priority? Yes, we do. However, the list of Inbound and Outbound rules are independent of each other. So, it’s ok to have matching rule priority numbers as the are not conflicting.

Tip: You can get your public IP address by executing:

curl ifconfig.me

Associating an NSG to NICs

While you technically can associate an NSG to a NIC it’s not advised. If you have a rule that is applicable to an associated subnet and another to an associated NIC, you do not get more security. The reason it’s advised to not associate an NSG to a NIC is simply to due to management of the rules. In a larger environment, you’re more likely to want to apply rules that affect multiple NICs on the same subnet. This is much easier than creating multiple rules that seem like duplicates each to a single NIC.

Note: Have you noticed that I refer to NICs and not VMs? This is because the NSG rules apply to all NICs even those of Private Endpoints.

Reply Traffic

The NSG is stateful meaning that it will allow (until traffic is idle long enough) a reply to a request without a rule for the opposite direction. For example, calls from a VM out to some location on the internet will have replies allowed in.

A Dirty Little Secret

When I was first learning about Azure networking I thought an NSG provided something like a perimeter around my VMs. Having a rule to deny traffic to a subnet meant I was scrutinizing the traffic to those VMs and that those VMs would be allowed to openly talk to each other. This is not the case at all!

The truth is, there is no virtual network! Every bit of information about VNet, subnet, routing, DNS, and NSG rules are explicitly applied to all of the NICs of the associated subnet(s). So when an NSG rule is for a subnet, it’s really being applied to each NIC on that subnet! Blocking SSH in a rule to a subnet means each NIC in that subnet has that block.

In reality, the NICs are simply programs running on an FPGA chip. Even the server that is “virtual” is made to believe, at the OS level, that the NIC is a real piece of hardware. It’s just code.

This also explains why you cannot simply move a VM to another VNet. You can change association to another subnet but to move to another VNet requires several underlying things to be recreated. At this time, it’s just not possible.

I was using the Azure Portal as I have done many times. Simply selected the resources, then Delete, applied ‘yes’ when prompted. After a few minutes they were all gone as expected. One day, one of the App Gateways with the required resources like Public Ip Address, Virtual Network, etc, were still there after an attempt to delete.

Selecting the App Gateway showed the details that included the IP address, version, etc. But, it also showed in a large bar: Failed. I have seen it show Deleting before but never failed. I selected the Delete option again and after many minutes nothing changed. So, I tried to delete it using PowerShell.

$gtw = Get-AzApplicationGateway -Name "dev-example-appgateway"
$gtw

Executing the two lines above showed the App Gateway’s Provisioning State as Failed and Operation State as Stopping.

I did some research and tried several things:

Start-AzApplicationGateway -ApplicationGateway $gtw
Stop-AzApplicationGateway -ApplicationGateway $gtw
Set-AzApplicationGateway -ApplicationGateway $gtw

Each took a few minutes and either did nothing or gave an error message. One message caught my eye.

1
2
3
/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/myResourceGroup1/providers/Microsoft.Network/publicIPAddresses/dev-example-public-ip used by resource

/subscriptions/ xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /resourceGroups/ myResourceGroup1/providers/Microsoft.Network/applicationGateways/dev-example-appgateway is not in Succeeded state. Resource is in Failed state.

Looking closely I noticed the issue may not be the App Gateway after all, but a dependent resource; Public Ip Address.

$pip = Get-AzPublicIpAddress -Name dev-example-public-ip -ResourceGroupName myResourceGroup1
$pip

Showing the details of the PIP also showed it was in a failed state.

All this time I thought it was the Application Gateway. So, with this extra knowledge, I tried a different approach. Some of the research suggested executing the Set command with no changes.

Set-AzPublicIpAddress -PublicIpAddress $pip
$pip

It worked! Well, at least for that resource. I tried the Set again for the App Gateway and it didn’t show a change. Instead, it gave the same error. Ok, let’s try the delete again just to see. Mind you, I have tried this command before including the -Force switch.

Remove-AzApplicationGateway -Force -Name dev-example-appgateway -ResourceGroupName myResourceGroup1

After a few minutes, it simply returned to a prompt. No error message this time. So, I went back to the Azure Portal and refreshed. It worked! Problem solved.

The significance here is that one resource can show a failed state when it’s really a dependent resource that is in trouble. I hope this helps someone else and you don’t spend the research time like I did.

Body

While learning Terraform some time back, I wanted to leverage Availability Zones in Azure. I was specifically looking at Virtual Machine Scale Sets. Terraform Azure Virtual Machine Scale Sets Looking at the documentation Terraform has, I noticed there is no good example on using zones. So, I tried a few things to see what was really needed for that field. While doing some research, I noticed there are many people in the same situation. No good examples. I figured I’d create this post to help anyone else. And, of course, it’s a good reminder for me too if I forget the syntax on how I did this.

Here’s a very simple Terraform file. I just created a new folder then a new file called zones.tf. Here’s the contents:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
variable "location" {
    description = "The location where resources will be created"
    default = "centralus"
    type = string
}

locals {
    regions_with_availability_zones = ["centralus","eastus2","eastus","westus"]
    zones = contains(local.regions_with_availability_zones, var.location) ? list("1","2","3") : null
}

output "zones" {
    value = local.zones
}

The variable location is allowed to be changed from outside the script. But, I used locals for variables I didn’t want to be changed from outside. I hard coded a list of Azure regions that have availability zones. Right now it’s just a list of regions in the United States. Of course, this is easily modifiable to add other regions.

The zones local variable uses the contains function to see if the specified region is in that array. If so, then the value is a list of strings. Else it’s null. This is important. The zones field in Azure resources required either a list of strings or null. An empty list didn’t work for me.

As it is right now, you can run the Terraform Apply command and you should see some output. Changing the value of the location variable to something not in the list and you may not see output at all simply because the value is null.

Now, looking at a partial example from the Terraform documentation:

1
2
3
4
5
6
7
resource "azurerm_virtual_machine_scale_set" "example" { 
    name = "mytestscaleset-1" 
    location = var.location
    resource_group_name = "${azurerm_resource_group.example.name}" 
    upgrade_policy_mode = "Manual" 
    zones = local.zones
}

Now the zones field can be used safely when the value is either a list of strings or null. After I ran the complete Terraform script for VM Scale Set, I went to the Azure Portal to verify it worked.

This proved to me that I can use a region with and without availability zones in the same Terraform script.

For a list of Azure regions with Availability Zones, see: Azure Availability Zones

The other day I was building out two Virtual Machines using an ARM template. I had been building many other VMs with no issues. Now, I have a failure. And two other VMs had issues being created. All four VMs had a Provision Status of Failed.

I checked the quota and determined that was not the issue. Looking at the failure message of each VM only said a failure occurred and provided no real details.

Why are you here?

This was ultimately fixed. This post is about a few things that occurred that allowed me to get these VMs out of a failed state.

The two VMs built using an ARM template worked successfully in another region. That other region had Availability Zones where the region with the failure did not. I did modify the ARM template to use Availability Sets instead.

The other two VMs were created by hand by a coworker. The steps used worked on other VMs in the same region. So, why the failures at all? Unfortunately, I don’t know. Fluke in the “matrix” for all I know. In those cases, were still stuck with determining the next step.

The Right SKU

One suggestion worked on the two VMs not created by the ARM template. I changed the size of the VM. They went from “Standard_F4s_v2” to “Standard_F4s”. After the size change, I was able to start the VMs. A coworker then confirmed he was able to install needed software.

I tried to change the size on the VMs created by the ARM template. But it failed to work. Why? Because I tried to delete the VMs so I could start over. That actually hurt me. I didn’t yet know about attempting to change the VM size. So, what finally fixed it for me?

Finally, My Answer

This info was what I got from a Microsoft ticket. The first round of discussions with them resolved nothing. It wasn’t until they realize another piece of my puzzle that they gave a different suggestion.

Our VMs were in Availability Sets. That alone is not an issue. The trick was to stop each VM in the Availability Sets. Once each VM was stopped I retried the delete of the problematic VMs. IT WORKED!! I was finally able to remove those VMs and recreate them.

Once recreated I then went to each of the other VMs and started them. They all come back online. Eureka!!!

Hopefully, this extra tidbit of detail will help you. If a VM fails on creation, don’t delete. Instead, try to change the VM size. If that fails, look at what else help dictate how a VM is allocated. For me, the Availability Set was the extra detail. Stopping all the VMs in an Availability Set allowed me to delete the problematic VMs and recreate them.