Cisco ACI Notes

Published by Keyboard Banger on

This is a collection of my Cisco ACI notes during my studies.

  • unenforced network: you get this type of network when you set the “Policy Control Enforcement Preference” to “Unenforced” at the VRF level. This disables the default security policiy on the VRF, which means all EPGs associated with the VRF can theoretically communicate together.
  • when companies migrate from a traditional network to ACI, they follow a staged approach. Often, they opt for the network-centric mode first. Second, they gradually migrate to the application-centric mode going through the hybrid mode. they can start by creating a Bridge Domain and subnet for every VLAN and put their servers that were in this VLAN on an EPG. Then they can gradually group those servers according to another criteria such as by application or by business need, rather than by VLAN as it was in the traditional network.
  • The network-centric mode
    • can be a one-tenant or a multi-tenant setup.
    • in a multi-tenant design, if you want your tenants to communicate in the future, then you must design non-verlapping subnets in them. You can not let overlapping subnets in your tenants and want to have communication between your tenants because in this case you need a certain address translation function between both tenants, and ACI does not support NAT to this day.
    • in this mode we take our existing VLANs and subnets from the old network and create them in ACI; i.e. we “reproduce” the network in ACI.
    • in a single tenant design, L2out and L3out can be placed in your tenant or in the tenant Common. But the recommendation is to place it in the tenant Common. Reason: you may want in the future to add further tenants and with the L2out and L3out already placed in the common tenant, you won’t be forced to redesign your tenants.
  • The hybrid mode is the combination of the network-centric mode and some features from the application-centric mode.
  • Blacklist model vs whitelist model: in an organisation the corporate security guidelines and policies follow one of the following security models: blacklist or whitelist. In a blacklist model everything is open unless specifically denied. In the whitelist model every communication is denied unless specifically authorized. A quick analogy to understand the whitelist model is Cisco IOS Access Lists. wh
  • L2out and L3out are L2 and L3 connections from ACI fabric to the core network of the company. In a multi-tenant design we can:
    • either attach one L3out connection to each tenant, or
    • we can centralize one L3out connection on tenant Common.
  • At the level of L2out and L3out:
    • ACI fabric makes adjacencies with the core network
    • we can control (allow or filter out) route prefixes into and out of the ACI fabric
  • ere an entry “deny any” at the end of the ACL. Cisco ACI employs the whitelist model by default.
  • A packet in the fabric is encapsulated within an UDP datagram
  • VXLAN in the ACI fabric is different from the standard VXLAN
  • VXLAN offers 16 Million subnets. Each segment is distinguished with a VNI VXLAN Network Identifier.
  • VNI defines a L2 broadcast segment or a L3 context
  • Unknown Unicast traffic is handeled in two ways
    • flood
    • Hardware Proxy: unknown unicast frames are forwarded to the spines, where the addresses are analyzed. If there is no match in the table the frames are dropped.
  • ARP traffic in the ACI fabric is handeled in two ways:
    • flood: MAC addresses are learned from the L2 traffic
    • Unicast routing: MAC addresses are learned from L2 traffic and IP addresses are learned from L3 traffic
  • Unknown Multicast traffic is a multicast traffic crossing the ACI fabric without a IGMP Join message.
  • VLAN allocation in APIC is either static or dynamic. Dynamic allocation is recommended.
  • To be able to logically connect a physical server to ACI fabric you need to configure a Physical Domain on APIC
  • To be able to logically connect a virtualized server to ACI fabric you nedd to configure a VMM Domain on APIC.
  • Physical Domain:
    • connects a physical external entity to the fabric
    • Cisco recommends one physical domain when connecting the ACI fabric to an external network
  • Virtual Domain:
    • connects a virtualized server to the fabric.
    • is configured on APIC as a VMM Domain.
  • Both Physical and Virtual Domains require VLAN pool(s) each.
  • One difference between virtual and physical Domains: a physical Domain does not use a virtual machine manager.
  • The VMM Domain
    • connects a Virtual Machine Manager such as Vmware vCenter or Microsoft Hyper-V to APIC.
    • once you create a VMM Domain on APIC, a virtual switch is created on the virtualized server.
    • must be assigned a VLAN pool. The VLAN pool will provide the VLAN IDs that will be assigned dynamically to Port Groups for example in case of Vmware integration with ACI.
  • When a physical server is connecting to the ACI fabric, then configure static VLAN mapping.
  • When a virtualized server is connecting to the ACI fabric, then use dynamic VLAN allocation.
  • Physical workload: is a subset of compute, storage and network ressources dedicated to a single entity. In the IT industry we distinguish physical workloads and virtual workloads. A Physical workload ist then the subset of compute, storage and network dedicated to physical machine(s). A Virtual Workload is the same subset being dedicated to virtual machine(s).
  • An external entity (router, switch, server…) is attached to the ACI fabric through:
    • one port
    • vPC
    • Port Channel
  • Bridge Domain
    • is a container of subnets, i.e. we define the IP subnets here.
    • we define subnets with their gateway IP addresses.
    • we can group subnets altogether in a same bridge domain, or separate them in different bridge domains. The second approach is necessary if you need to place firewall policies between subnets.
    • points to one and only one VRF.
    • can be configured without a VRF (APIC GUI allows it), which can be added later.
    • on the APIC, it is configured under Tenant -> Networking.
    • can be created through context menus (on the left) or graphically with drag and drop. If there is more than one VRF already created, then you must pay attention while dragging the bridge domain symbol: you must release the bridge domain symbol over the desired VRF.
  • AEP is required for attaching the external entity to the fabric.
  • AEP: Attach Entity Profile (see how to configure AEP in ACI)
    • links an infrastructure policy group to fabric interface(s), where an external entity connects. External entities with similar infrastructure policy requirements should be assigned the same AEP.
    • encapsulates one or more Domains.
    • requires a VLAN Pool
  • Tenants
    • provide security by isolating what is defined under tenant A from tenant B.
    • With tenants we can run many logical networks on the same physical network
    • provide a separation of the control and management plane, ie each tenant has its own control plane and management plane.
    • There are various tenant design possibilities. We can design our fabric to run with a single tenant or with many. Possible criteria for a multi-tenant design, can be:
      • department naming: Sales, Engineering, Tests
      • location in the product lifecycle: Production, Test-Env,
      • security zones: standard, DMZ,…
      • etc.
    • There are default tenants already preconfigured on the APIC:
      • management tenant
      • infrastructure tenant
      • common tenant: here we can define common network policies and services that will be used across tenants. Some of these services could be DHCP, DNS, Active Directory, etc.
    • in ACI, management of tenants can be performed on a per-tenant basis. And we can assign tenant management on a user or group basis.
  • Infrastructure administrator vs tenant administrator
    • Infrastructure administrator manages and controls VLAN namespaces for all tenants. He has access to all tenants.
    • Tenant administrator has access only to his allowed tenant(s) and his/their ressources.
  • NTP must be configured and synchronized on APIC and all fabric nodes.
  • New nodes being added to the fabric are automatically discovered by APIC through LLDP. As soon as they pop up in the APIC GUI Interface you can add or block them from joining the fabric, based on their Serial Numbers.
  • New fabric nodes send DHCP requests and receive replies from APIC.
  • APIC sends TEP addresses to the new leafs
  • Giving lower numerical IDs to the spines is recommended. The subsequent higher IDs should be reserved for the leafs.
  • All fabric nodes and APICs should be connected to an OOB network for management purposes.
  • Access to leaf switches through console cable is possible but offers only read capabilities.
  • OS image management occurs on the APIC, which supports TFTP
  • in ACI there is no need to:
    • configure loopback addresses on new switches
    • configure IGP protocol and neighborships
    • configure custom routing timers
    • configure list of allowed VLANs on trunks.
  • Swtiches in a Pod share the same VTEP prefix
  • VRF, aka private network:
    • pronounced V-R-F or way fancier “Vurf” :)
    • is equivalent to the legacy VRF concept.
    • is pointed by one or more bridge domains
    • can be created with context menus or graphically with drag and drop
    • you can configure the VRF on ACI first or the bridge domain first. The order does not matter because you can attach the bridge domain to the VRF later.
    • Subnets must be unique only VRF-wide. It means:
      • you can’t have a subnet Subnet_S1 on two bridge domains belonging to the same VRF
      • VRF_A and VRF_D can both have a subnet with the same subnet range of IP addresses.
    • APIC can be instructed to confine the subnets within a given VRF, or to propagate them to other VRFs, or to allow redistributing them to the outside network through a L3out connection.
    • APIC automatically creates an Infrastructure VRF to communicate with fabric switches
  • Management of the fabric can be performed also using an external management station connected to the fabric on tenant “mgmt”. In this scenario you must:
    • configure a VLAN Pool, an AEP, a phyiscal domain
    • assign the VLAN Pool to the domain
    • encapsulate the domain under the AEP
  • Application Network Profile, aka ANP or Application Profile
    • must be configured before even creating EPGs
    • while creating the Application Network Profile you can create EPGs as well. Or you can leave creating EPGs later on.
  • End Point Groups, aka EPG
    • while creating an EPG, a bridge domain must be associated to it.
    • on the APIC GUI, using the Topology tab: after dropping an EPG symbol in the window and configuring it, it will not be created unless you press the Submit button.
  • Provisioning a switch port in traditional networks is completely different from the ACI world:
    • in a traditional switch you configure interfaces separately
    • in ACI, you configure many constructs and objects at first, sch as domain, AEP, VLAN Pool, Switch Profile, Interface Profile… which may seem a burden at first. But its power lays with its flexibility and extensibility. For example if you want to add an interface with similar configuration to a previous one, simply add it to the Interface Profile.
  • an Application in the ACI model ist not a virtual/physical machine, but the combination of:
    • workloads, either physical or virtual
    • L2 – L7 policies: VLANs, subnets, L4 ports, ACL, QoS policies, filtering policies, load balancing policies,…
  • ACI fabric contains 2 to 6 spines: 2, 4, 6.
  • ACI fabric operates on a whitelist model: no communication is allowed unless specified.
  • Frames in ACI are routed, but the L2 switching semantics are preserved.
  • Contracts:
    • policies i.e rules intended to regulate the communication between EPGs. The policies include one or more of the following constructs:
      • permit
      • deny
      • log
      • mark
      • QoS
      • redirect
      • service graph
    • can be as simple as one rule or complex.
    • complex contracts contain Subjects, Labels, Filters and Actions.
    • a Subject can be seen as a function. An example of this would be a web server providing HTTP, HTTPS and FTP services.
    • can be grouped together in a bundle; an EPG providing the “services” ist said to be a Provider, or Provider EPG. The EPG benefitting from the “services” ist said to be a Consumer, thus a Consumer EPG.
    • can be leveraged by more than one EPG, i.e an EPG A and an EPG B point to the same Contract.
    • can be defined globally on the fabric, or under a tenant.
    • a particular kind of contracts is called Taboo. A Taboo defines a list of deny actions and is applied to an EPG, i.e that EPG will be denied access to these ressources. Taboos enforce the black list model which I described in the beginning of this post. Taboos are especially used by clients that are using the blacklist model already and performing a migration to ACI infrastructure.
  • L4-7 Service insertion
    • is the process of introducing L4-7 services in the data path of a packet in ACI fabric
    • requires a service graph and the mapping of it to concrete devices
    • in ACI we can choose one of three L4-7 Service Insertion modes:
      • with device package: this is the full integration of the L4-7 device with ACI fabric.
      • service manager mode: security policies are defined on the L4-7 device by its own administrator. Then the policies are integrated and orchestrated by ACI
      • no device package or service manager: the L4-7 device is completely managed by its own administrator. ACI administrator only creates the necessary VLANs. This modes is helpful in “political” environments, where each department still wants to completely manage its equipement.
  • Service graph
    • is a collection of Abstract Nodes
    • is connected to EPG with a contract
  • Meta device
    • is a symbolic representation of the L4-7 device that is connecting to the fabric
    • its function, whether it is a firewall, or a load balancer or else, is defined in the Abstract Node
  • Logical Device (LDev)
    • is a cluster of two or more Concrete Devices
  • Concrete Device (CDev)
    • is a one-to-one representation of the L4-7 device
    • Interfaces of the Concrete Device (vnsCIf) represent one-to-one the interfaces of the real device but in the format {slot_port}, like interface 1_2 or interface 1_4
    • Each Interfac on the Concrete Device maps to an interface on the Logical Device (LIf)

I’ve distilled all these Notes from:

Categories: Cisco ACI

Keyboard Banger

Keyboard Banger is a network engineer from Africa. He has been working in network support and administration since 2008. He started writing study notes about certification exams and technology topics a couple of years ago. When he's not writing articles, he can be found wandering on technical forums.

0 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *