Network Isolation Pipeline

[nitindagar0]

Context:

In the MP clusters, NSG (Network Security Group) rules are currently managed through the NI (Network Isolation) controller. The NI controller is responsible for creating and updating NSG resources to ensure network isolation in AKS clusters. This discussion outlines a new approach to streamline the rollout of NSG rules in MP clusters by leveraging an inventory-based configuration to support both platform and partner workload rules. This solution aims to reduce deployment time, enable better support for sovereign clouds, and ensure a structured deployment process.

Problem Statement:

1. Long Deployment Time: Since NSG changes are bundled with Nodepool upgrades, it is difficult to decouple infrastructure updates from security rule changes. This integration complicates the process and often leads to delays, as the NSG rollout cannot proceed independently from the node upgrade process.
2. Limited Scope: It supports only platform rules and does not manage partner workload rules. As a result, namespaces that require workload-specific rules cannot be onboarded to sovereign clouds like France and Germany.
3. Lack of Firewall Rules: The current NI controller does not support firewall rules, which are necessary for enhanced network security.
4. Configuration Management: NSG rules persisted within the NI controller's pod instead of an external inventory, making it difficult to update rules. Changes require redeploying the NI controller, leading to long deployment cycles.

Possible Solutions

1. By Creating new NI pipeline similar to NP pipeline.
2. Publishing data using inventory publisher.
3. Using ECS to maintain NI rules.

2) Publishing data using inventory publisher.

• Partner will raise a PR and merge its changes in GriffinD2 repository.

• Whenever any partner change is merged into the master branch of GriffinD2, it triggers an ADO release pipeline named "NI pipeline".

• The pipeline would copy all the config files to a blob.

• Our Inventory publisher will monitor the Blob change feed, if there is an update, then Inventorypublisher will be triggered to consume the latest file.

• The Inventory publisher will send the platform and partner workload level NSG rules to the CA Inventory. For each workload, it will simply send a request to update the existing object already present on the cluster.
  Inventory publisher will create separate table for both platform and workload rules like griffinD2 repo
  Platform NI Rules
  Workload NI Rules

  Following Spec will be created once inventory publisher generates resources

• Whenever the cluster controller will be performing the reconciliation for a cluster resource it will do the following: -
  Fetch Platform rules which are at Ring level.

• Get all the namespace instances that should be deployed on the cluster.

• For each namespace instance, get its workload.

• It will generate NI rules as per following spec.

• Merge all rules for each current cluster and populate “NI Security Rules” Table.

• NSG service will fetch Data from NI rules and CRDs and will apply Update NSG table.

• NSG Table in Inventory is the source of truth for NSG rules along with firewall rules

• NI reconciler will consume NSG table to update NSG resources.

• Update Azure NSG resources.