Designing the network routing, cloud & hybrid, on a Hyperscaler like Microsoft Azure can get a real pain. I got an idea, to start playing with some common scenarios and make some notes, in order to have it as a reference. I end up with three scenarios. Those scenarios are referring to internal network topology, without public access, and I focus more on the routing aspect, the DNS configuration & inspection through Azure Firewall.
- Scenario 1: Hybrid connectivity with Azure VPN Gateway
- Scenario 2: Hybrid connectivity with Azure VPN Gateway & Traffic Inspection with Azure Firewall
- Scenario 3: Hybrid Connectivity with Azure VPN Gateway, second level peered networks & full traffic inspection with Azure Firewall
This scenario shares the same topology as Azure Routing Experiences | Scenario 1 with some changes in order to route all traffic through the Azure Firewall. This will allow us to have full control & inspection of all traffic from/to on-premises and all Azure traffic from/to all VNets. To achieve this we need some Route Table configuration.
At first, to control the traffic from the on-premises network, we need to add a Route Table to the GatewaySubnet to route all traffic through the Azure Firewall. 192.168.0.0/24, 192.168.4.0/24 & 192.168.5.0/24 Next Hop: 192.168.2.4. Now if we try to access the Storage Account, we will realize that we are still going directly from the VPN Gateway to the Storage account, (10.0.1.4 -> AzureGW -> 192.168.4.4). This happens because the Private Endpoint services populate a /32 route to the VNet and all peered VNets. The effective routes of the VPN Gateway now have our custom 192.168.4.0/24 -> 192.168.2.4 route but they also have the 192.168.4.4/32 -> Virtual Network route that is created by the Private Endpoint Service. Since it is a more specific route, it takes priority. The only way to bypass this in a hybrid scenario like this is to create a /32 route “192.168.4.4/32 Next Hop 192.168.2.4”. You can check more scenarios here. Now the GatewaySubnet Route Table is:
192.168.0.0/24 Next Hop 192.168.2.4
192.168.4.0/24 Next Hop 192.168.2.4
192.168.5.0/24 Next Hop 192.168.2.4
192.168.4.4/32 Next Hop 192.168.2.4
After this Route Table is applied to the GatewaySubnet, we can inspect the traffic coming from on-premises through the Azure Firewall logs. The /32 Route for the Private Endpoint must be applied to all Route Tables that are on VNets that are directly peered with the Private Endpoint VNet. I also added the /32 route to the VMSubnet. The Spoke2 subnet does NOT need to have the /32 route, since it is not directly peered with the Private Endpoint VNet. To access the Private Endpoint we added the /32 route to all Hub VNets, the GatewaySubnet, and the VMs subnet. The /32 route is not needed only services at the Spoke2 subnet, that is not directly peered with the Spoke1 subnet that has the Private Endpoint. And this is why the /32 route is created together with the Private Endpoint and it is populated to all peered subnets.
About the DNS configuration, as discussed in Scenario 1, to access Azure PaaS services, like the Storage, the Azure SQL, the Web App, we need to use the URI, since they don’t listen to the IP. Since we have Azure Firewall at the configuration, and I have enabled the Azure Firewall DNS Proxy feature to use it as a Conditional Forwarder for the on-premises DNS, we can also use it for the Azure estate. I changed the DNS of the Spoke2 VNet to the Azure Firewall, 192.168.2.4 to be able to resolve the Storage account’s Private IP. Linking all Private DNS zones to Azure Firewall’s VNet, usually the Hub VNet, we can use Azure Firewall as our DNS server to all VNets.
DNS & Routing example
DNS: The on-premises Server X, 10.0.2.10, makes a request to https://azappsa.blob.core.windows.net. At first, it asks the DNS to resolve the URL to an IP. The DNS has a conditional forwarder about blob.core.windows.net, and asks the Azure Firewall, 192.168.2.4. Azure Firewall has a linked Private DNS zone that has a host record for azappsa.blob.core.windows.net and it resolves to 192.168.4.4. This information routes back to Server X. Now Server X knows that the IP address of azappsa.blob.core.windows.net is 192.168.4.4.
Routing: To go to 192.168.4.4 first it asks its Default Gateway, in our case the RRAS. The RRAS has a custom route for 192.168.0.0/16 and forwards the packet to the VPN interface. The packet reaches the Azure VPN Gateway. The Azure VPN Gateway has a custom route for 192.168.4.4/32 and sends the packet to the Azure Firewall, 192.168.2.4. The Azure Firewall does not have a custom route, but it has a route that is automatically populated by the VNet peering with address 192.168.4.4/32. The Azure Firewall knows to forward the packet through the VNet peering and reaches the destination.
You can find more commends and test at the diagram below:
References:
Azure Routing Experiences | Scenario 1 – Apostolidis Cloud Corner
Use Azure Firewall to inspect traffic destined to a private endpoint – Azure Private Link | Microsoft Docs
Azure Private Endpoint DNS configuration | Microsoft Docs
What is a virtual network link subresource of Azure DNS private zones | Microsoft Docs
Azure Firewall DNS Proxy details | Microsoft Docs
Create, change, or delete an Azure route table | Microsoft Docs
Pantelis Apostolidis is a Sr. Specialist, Azure at Microsoft and a former Microsoft Azure MVP. For the last 20 years, Pantelis has been involved to major cloud projects in Greece and abroad, helping companies to adopt and deploy cloud technologies, driving business value. He is entitled to a lot of Microsoft Expert Certifications, demonstrating his proven experience in delivering high quality solutions. He is an author, blogger and he is acting as a spokesperson for conferences, workshops and webinars. He is also an active member of several communities as a moderator in azureheads.gr and autoexec.gr. Follow him on Twitter @papostolidis.
Hi Pantelis Apostolidis, Excellent write-up, thank you for sharing detailed info. Do we need to create individual route entries for all spoke subnets in the gateway route table or can we add single entries like 192.168.4.0/23 to NVA instead of 192.168.4.0/24 and 192.168.5.0/24 to NVA (192.168.2.4)
Hi Leo, the GatewaySubnet learns all other subnets of its own VNET and all subnets of all peered VNETS and those subnet address spaces are added to the GatewaySubnet system routes automatically. You can check this by viewing the Effective routes of the GatewaySubnet route table. So you need to overwrite those routes with your user routes. To do this you need to add same or more specific routes. The /23 is a larger route than the /24. So the /24 system route will take priority.
in short, yes you need to add the specific /24 routes.