private endpoints Archives - Apostolidis Cloud Corner

Azure Routing Experiences | Scenario 3

Pantelis Apostolidis — Sun, 06 Feb 2022 19:26:16 +0000

Designing the network routing, cloud & hybrid, on a Hyperscaler like Microsoft Azure can get a real pain. I got an idea, to start playing with some common scenarios and make some notes, in order to have it as a reference. I end up with three scenarios. Those scenarios are referring to internal network topology, without public access, and I focus more on the routing aspect, the DNS configuration & inspection through Azure Firewall.

At the previews posts, we covered the basics of routing traffic from/to on-premises, inspecting all traffic through Azure Firewall, and configuring the DNS for accessing the Private Endpoints. In this scenario, I am experimenting with connectivity between on-premises, the Hub & Spoke networks and a second level peered network (a network that is peered behind the Spoke network).

Recap of Scenario 1 & 2: We have a Hub network, two Spoke networks and an IPSec VPN connection with my on-premises network. We established routing all traffic through the Azure Firewall for inspection & configured DNS for accessing the Private Endpoint from on-premises & all Azure VNets.

In the third scenario, I am adding a new Spoke VNet, the “Azure 2” peered with my hub, and a third VNet, the “Azure 3” that is only peered with the “Azure 2” VNet. To enable connectivity between the “Azure 3” VNet and the rest of the networks, including the on-premises, we need a router at the “Azure 2” VNet. This can be an NVA or Azure Firewall. In my case, I added an Azure Firewall. The Azure Firewall of “Azure 2” VNet has the private IP: 192.168.200.64.

Azure 3 VNet, VM Subnet Route Table:
- 10.0.0.0/16 NH 192.168.200.68
- 192.168.0.0/20 NH 192.168.200.68
- 192.168.4.0/24 NH 192.168.200.68
- 192.168.5.0/24 NH 192.168.200.68
- 192.168.200.0/28 NH 192.168.200.68

Azure 2 VNet, AzureFirewallSubnet:
- 10.0.0.0/0 NH Internet
- 10.0.0.0/16 NH 192.168.2.4
- 192.168.0.0/20 NH 192.168.2.4
- 192.168.4.0/24 NH 192.168.2.4
- 192.168.5.0/24 NH 192.168.2.4

Azure 2 VNet, VM Subnet:
- 10.0.0.0/16 NH 192.168.200.68
- 192.168.4.0/24 NH 192.168.200.68
- 192.168.5.0/24 NH 192.168.200.68
- 192.168.0.0/20 NH 192.168.200.68
- 10.100.0.0/16 NH 192.168.200.64

Azure Hub VNet, AzureFirewallSubet:
- 10.0.0.0/0 NH Internet
- 192.168.200.0/28 NH 192.168.200.68
- 10.100.0.0/16 NH 192.168.200.68
Azure Hub VNet, VM Subnet:
- 10.0.0.0/16 NH 192.168.2.4
- 192.168.4.0/24 NH 192.168.2.4
- 192.168.5.0/24 NH 192.168.2.4
- 192.168.4.4/32 NH 192.168.2.4
- 192.168.200.0/24 NH 192.168.2.4
- 10.100.0.0/16 NH 192.168.2.4
Azure Hub VNet, GatewaySubnet:
- 192.168.0.0/24 NH 192.168.2.4
- 192.168.4.0/24 NH 192.168.2.4
- 192.168.5.0/24 NH 192.168.2.4
- 192.168.4.4/32 NH 192.168.2.4
- 192.168.200.0/24 NH 192.168.2.4
- 10.100.0.0/16 NH 192.168.2.4
Spoke 2 VNet, VM Subnet:
- 10.0.0.0/16 NH 192.168.2.4
- 192.168.4.0/24 NH 192.168.2.4
- 192.168.0.0/24 NH 192.168.2.4
- 192.168.200.0/24 NH 192.168.2.4

Routing Example

Let’s describe a packet’s journey. The On-premises Server X (10.0.2.10) makes sends a packet to 10.100.0.4. 1st hop the packet goes to the default gateway, reaching the on-premises VPN device, in our case the RRAS. The RRAS has a custom route for 10.100.0.0/16 and forwards the packet to the VPN interface. The packet reaches the Azure VPN Gateway The Azure VPN Gateway has a custom route for 10.100.0.0/16 and forwards the packet to the HUB Azure Firewall, 192.168.2.4. The HUB Azure Firewall has a custom route for 10.100.0.0/16 and forwards the packet to the “Azure 2” Azure Firewall, 192.168.200.68. The “Azure 2” Azure Firewall does not have a custom route, but it has a route for 10.100.0.0/16 that is automatically populated by the VNet peering. The Azure FIrewall knows to forward the packet through the VNet peering and reaches the destination.

You can find more commends and tests in the below diagram with the whole solution.

Diagram: (Click here to download a high-resolution SVG image)

References:
Azure Routing Experiences | Scenario 1 – Apostolidis Cloud Corner
Azure Routing Experiences | Scenario 2 – Apostolidis Cloud Corner
Use Azure Firewall to inspect traffic destined to a private endpoint – Azure Private Link | Microsoft Docs
Azure Private Endpoint DNS configuration | Microsoft Docs
What is a virtual network link subresource of Azure DNS private zones | Microsoft Docs
Azure Firewall DNS Proxy details | Microsoft Docs
Create, change, or delete an Azure route table | Microsoft Docs

Pantelis Apostolidis

Pantelis Apostolidis is a Sr. Specialist, Azure at Microsoft and a former Microsoft Azure MVP. For the last 20 years, Pantelis has been involved to major cloud projects in Greece and abroad, helping companies to adopt and deploy cloud technologies, driving business value. He is entitled to a lot of Microsoft Expert Certifications, demonstrating his proven experience in delivering high quality solutions. He is an author, blogger and he is acting as a spokesperson for conferences, workshops and webinars. He is also an active member of several communities as a moderator in azureheads.gr and autoexec.gr. Follow him on Twitter @papostolidis.

www.cloudcorner.gr

The post Azure Routing Experiences | Scenario 3 appeared first on Apostolidis Cloud Corner.

Azure Routing Experiences | Scenario 2

Pantelis Apostolidis — Sun, 06 Feb 2022 19:13:24 +0000

This scenario shares the same topology as Azure Routing Experiences | Scenario 1 with some changes in order to route all traffic through the Azure Firewall. This will allow us to have full control & inspection of all traffic from/to on-premises and all Azure traffic from/to all VNets. To achieve this we need some Route Table configuration.

At first, to control the traffic from the on-premises network, we need to add a Route Table to the GatewaySubnet to route all traffic through the Azure Firewall. 192.168.0.0/24, 192.168.4.0/24 & 192.168.5.0/24 Next Hop: 192.168.2.4. Now if we try to access the Storage Account, we will realize that we are still going directly from the VPN Gateway to the Storage account, (10.0.1.4 -> AzureGW -> 192.168.4.4). This happens because the Private Endpoint services populate a /32 route to the VNet and all peered VNets. The effective routes of the VPN Gateway now have our custom 192.168.4.0/24 -> 192.168.2.4 route but they also have the 192.168.4.4/32 -> Virtual Network route that is created by the Private Endpoint Service. Since it is a more specific route, it takes priority. The only way to bypass this in a hybrid scenario like this is to create a /32 route “192.168.4.4/32 Next Hop 192.168.2.4”. You can check more scenarios here. Now the GatewaySubnet Route Table is:
192.168.0.0/24 Next Hop 192.168.2.4
192.168.4.0/24 Next Hop 192.168.2.4
192.168.5.0/24 Next Hop 192.168.2.4
192.168.4.4/32 Next Hop 192.168.2.4

After this Route Table is applied to the GatewaySubnet, we can inspect the traffic coming from on-premises through the Azure Firewall logs. The /32 Route for the Private Endpoint must be applied to all Route Tables that are on VNets that are directly peered with the Private Endpoint VNet. I also added the /32 route to the VMSubnet. The Spoke2 subnet does NOT need to have the /32 route, since it is not directly peered with the Private Endpoint VNet. To access the Private Endpoint we added the /32 route to all Hub VNets, the GatewaySubnet, and the VMs subnet. The /32 route is not needed only services at the Spoke2 subnet, that is not directly peered with the Spoke1 subnet that has the Private Endpoint. And this is why the /32 route is created together with the Private Endpoint and it is populated to all peered subnets.

About the DNS configuration, as discussed in Scenario 1, to access Azure PaaS services, like the Storage, the Azure SQL, the Web App, we need to use the URI, since they don’t listen to the IP. Since we have Azure Firewall at the configuration, and I have enabled the Azure Firewall DNS Proxy feature to use it as a Conditional Forwarder for the on-premises DNS, we can also use it for the Azure estate. I changed the DNS of the Spoke2 VNet to the Azure Firewall, 192.168.2.4 to be able to resolve the Storage account’s Private IP. Linking all Private DNS zones to Azure Firewall’s VNet, usually the Hub VNet, we can use Azure Firewall as our DNS server to all VNets.

DNS & Routing example

DNS: The on-premises Server X, 10.0.2.10, makes a request to https://azappsa.blob.core.windows.net. At first, it asks the DNS to resolve the URL to an IP. The DNS has a conditional forwarder about blob.core.windows.net, and asks the Azure Firewall, 192.168.2.4. Azure Firewall has a linked Private DNS zone that has a host record for azappsa.blob.core.windows.net and it resolves to 192.168.4.4. This information routes back to Server X. Now Server X knows that the IP address of azappsa.blob.core.windows.net is 192.168.4.4.

Routing: To go to 192.168.4.4 first it asks its Default Gateway, in our case the RRAS. The RRAS has a custom route for 192.168.0.0/16 and forwards the packet to the VPN interface. The packet reaches the Azure VPN Gateway. The Azure VPN Gateway has a custom route for 192.168.4.4/32 and sends the packet to the Azure Firewall, 192.168.2.4. The Azure Firewall does not have a custom route, but it has a route that is automatically populated by the VNet peering with address 192.168.4.4/32. The Azure Firewall knows to forward the packet through the VNet peering and reaches the destination.

You can find more commends and test at the diagram below:

References:
Azure Routing Experiences | Scenario 1 – Apostolidis Cloud Corner
Use Azure Firewall to inspect traffic destined to a private endpoint – Azure Private Link | Microsoft Docs
Azure Private Endpoint DNS configuration | Microsoft Docs
What is a virtual network link subresource of Azure DNS private zones | Microsoft Docs
Azure Firewall DNS Proxy details | Microsoft Docs
Create, change, or delete an Azure route table | Microsoft Docs

Pantelis Apostolidis

www.cloudcorner.gr

The post Azure Routing Experiences | Scenario 2 appeared first on Apostolidis Cloud Corner.

Azure Routing Experiences | Scenario 1

Pantelis Apostolidis — Sun, 06 Feb 2022 18:41:14 +0000

Scenario 1: Hybrid connectivity through Azure VPN gateway

At the start, I created the on-premises network, using a RRAS (Windows Server 2019 with Routing & Remote Access) to act as the router/VPN device and a Server with DNS service. The Azure estate has three VNets, in a hub & spoke topology. One HUB and two Spokes, connected with the HUB with VNet peering. There is no peering between the spokes. My Azure HUB network has three subnets, one has a VPN Gateway, the second has a VM, and the third has an Azure Firewall. In this first scenario, the Azure Firewall acts only as a DNS Proxy. It is not included in routing.

On-premises network: 10.0.0.0/16
HUB VNet Address Space: 192.168.0.0/22
HUB GatewaySubnet: 192.168.1.0/24
HUB VMSubnet: 192.168.0.0/24
HUB Firewall Subnet: 192.168.2.0/24
Spoke1 (storage account): 192.168.4.0/24
Spoke2 (VM): 192.168.5.0/24

The first spoke has a Private Link to my storage account. The second spoke has a VM.. Azure VPN Gateway knows all routes of its VNet, the peered VNets & the routes propagated from the VPN connection. From on-premises, we can reach all resources using the VPN connection interface ( in RRAS I added a custom route “192.168.0.0 255.255.0.0 interface:AzureGW”)

From the VM of the HUB VNet (192.168.0.4), in order to be able to reach the on-premises network, we need a custom route, since the on-premises network is not populated to the VNet. I created a Route Table, with route “10.0.0.0/16 Next Hop: Virtual Network Gateway” attached to the VM Subnet.

To access the Private Endpoint of the Storage account we need some more resources & configuration. The storage account, as all PaaS services (like Web App & Azure SQL) responds only to URI and not to IP. Since we have connected a Private Endpoint to the Storage Account, the Public Access is blocked. So, in order to connect to the storage account, we need the aprostore.file.core.windows.net to translate to the private IP of the storage account, the 192.168.4.4. The proper way to achieve this is by using DNS.

First, we need to create a Private DNS zone and link it to the HUB VNET. For accessing blob storage we need a Private DNS zone with the name privatelink.blob.core.windows.net, for the file we need privatelink.file.core.windows.net. More services here. Then add the Storage Account Private Endpoint record to the Private DNS Zone. Now there is an A record azappsa with IP 192.168.4.4. Now, all Azure resources at the linked VNet, the HUB, are able to resolve the DNS records of the Private DNS Zone. We cannot resolve the records of the Azure Private DNS Zone from on-premises. To do so, we need a DNS server on Azure, to use as a conditional forwarder. This can be a Windows or Linux VM with DNS services or in my case, the Azure FIrewall with the DNS proxy enabled (I will use the whole functionality of the firewall to my next scenarios). I enabled the DNS Proxy on Azure, using default Azure DNS, and I added a conditional forwarding at the on-premises DNS “blob.core.windows.net -> 192.168.2.4” and “file.core.windows.net -> 192.168.2.44”. Now, I can successfully resolve the private IP of the storage account using its name, and be able to connect to it to both blob and to files with SMB access.

From the VM of the Spoke2 VNet (192.168.5.4), in order to be able to reach the on-premises network, we need a custom route, since the on-premises network is not populated to the VNet. I created a Route Table, with route “10.0.0.0/16 Next Hop: Virtual Network Gateway” attached to the VM Subnet. I added a second route “192.168.4.0/24 Next Hop: Virtual Network Gateway” & changed the VNet DNS to 192.168.2.4 (the Azure Firewall) to be able to access the Spoke1 VNet for storage access.

DNS & Routing example

Routing: To go to 192.168.4.4 first it asks its Default Gateway, in our case the RRAS. The RRAS has a custom route for 192.168.0.0/16 and forwards the packet to the VPN interface. The packet reaches the Azure VPN Gateway. The Azure VPN Gateway has a custom route for 192.168.0.0/24 BUT it also has a route for 192.168.4.4/32 that is automatically populated by the VNet peering. The /32 route is more specific than the /24 route, so the VPN Gateway forwards the packet directly to the Private Endpoint, bypassing the Azure Firewall. (At the Azure Routing Experiences | Scenario 2 we will see how we will force the traffic through the Azure Firewall.

Please find below the whole solution diagram, I tried to make it as analytic as possible, without messing with too many lines. Also, I have some notes and tests below.

References:
Azure Private Endpoint DNS configuration | Microsoft Docs
What is a virtual network link subresource of Azure DNS private zones | Microsoft Docs
Azure Firewall DNS Proxy details | Microsoft Docs
Create, change, or delete an Azure route table | Microsoft Docs

Pantelis Apostolidis

www.cloudcorner.gr

The post Azure Routing Experiences | Scenario 1 appeared first on Apostolidis Cloud Corner.