Multicast routing design - PIM, RP, MSDP

Requirement / business driver

Deliver one-to-many (or many-to-many) efficiently with the right tree type and a resilient rendezvous-point (RP) design.

PIM modes

RPF - the multicast forwarding rule

Multicast forwarding is governed by the Reverse Path Forwarding (RPF) check: a router forwards a multicast packet only if it arrived on the interface that unicast routing uses to reach the source - otherwise it silently drops it (this is what prevents multicast loops). RPF fails wherever the multicast-enabled topology diverges from the unicast best path. Fixes: realign multicast interfaces with unicast, add a static mroute, or carry multicast RPF routes in MP-BGP (Multicast BGP). Design rule: pick the fix that avoids RPF failure with the least added control-plane complexity - if BGP is already the unicast protocol, MP-BGP adds no new protocol.

Application fit for PIM modes: SSM for known/single sources (IPTV, market data - no RP, simplest); PIM-BIDIR for many-to-many (classic financial apps); PIM-SM for general groups; PIM-DM (dense flood-and-prune, no RP) is legacy - avoid in new designs.

Group addressing and scoping

Plan multicast group addresses like unicast addresses: a schema, rigorously controlled, because summarization matters for multicast too (Route aggregation and summarization) and uncontrolled assignment breeds conflicts and meaningless groups. The administratively scoped block 239.0.0.0/8 (16M+ groups) is reserved for internal use - carve it into per-scope and per-application ranges (site-local, campus, enterprise) and enforce the plan with multicast boundaries at the edges of each scope, so internal groups never leak out and external groups never enter. IPv6 builds this in: every group address carries a scope field, so scoping is native rather than boundary-filtered.

RP selection and placement (for PIM-SM)

• Static RP, Auto-RP, or BSR to distribute RP info.
• Anycast-RP (with MSDP or PIM) gives RP redundancy and load-sharing - the standard resilient design.
• Place RPs near sources or in the core; size for the group/source count.

MSDP

Connects PIM-SM domains for inter-domain source discovery, and underpins intra-domain Anycast-RP.

RP redundancy techniques

• Anycast-RP (with MSDP) - two or more RPs share one loopback /32; IGP steers each source/receiver to the nearest RP, and MSDP between the RPs exchanges active-source (SA) state (a source may register with one RP while receivers join another). On RP failure, IGP reconverges to a surviving RP. IPv4 only; MSDP is TCP-based and its SA cache aids troubleshooting.
• Anycast-RP using PIM (RFC 4610) - same goal without MSDP: each RP forwards copies of PIM Register messages to its peer RPs, so every RP learns all sources. Works for both IPv4 and IPv6 (IPv6 has no MSDP). MSDP may still be needed for IPv4 interdomain source exchange.
• Phantom RP - RP redundancy for PIM-BIDIR (everything rides the shared tree, so RP HA is critical). The RP is a routable address, not a physical router: two routers advertise the same RP subnet with different mask lengths, IGP prefers the longest match as the active root, and failover to the shorter-mask path follows IGP convergence.
• RP discovery: Auto-RP (Cisco-proprietary, can fall back to dense) vs BSR (RFC 5059, PIMv2 standard) - don't run both in one domain (except at a boundary for PIMv1/PIMv2 interop).

Live-Live (zero-loss multicast)

For applications that tolerate no loss and no retransmission - classically financial market data, where milliseconds cost money - send two simultaneous copies of the stream to two different group sets over physically disjoint paths/infrastructure, and let the receiver take whichever arrives cleanly. A single shared core with MPLS-TE FRR and SRLG-diverse paths can approximate this but isn't true physical separation. It's the multicast analogue of dual-plane "send-two-copies" core resilience (Network hierarchy and topologies).

MoFRR - make-before-break inside the network

Multicast-only Fast ReRoute moves the live-live idea inside the network: the egress router joins the same tree twice, sending primary and backup joins up two IGP-disjoint (ECMP/UCMP) paths toward the source or root. Both copies flow; the egress forwards from the primary only and discards the backup at line rate, flipping to the already-flowing backup the moment the primary path drops - RIB-driven, restoration in the low hundreds of milliseconds, far faster than tree reconvergence though short of true link FRR. It works for PIM trees and for mLDP (in-band-signalled trees only - the egress must be able to originate the second tree itself), and its hard precondition is real multipath toward the root: no ECMP, no MoFRR. Against Live-Live above, the contrast is scope: live-live is end-to-end and application-level (two streams, the receiver merges), MoFRR is network-internal (one stream delivered; the duplication - and its bandwidth cost - is contained between the divergence point and the egress).

FHRP-aware PIM

PIM DR election is independent of FHRP, so on a gateway pair running HSRP/VRRP the active FHRP router and the PIM DR may not be the same device - multicast can black-hole on failover. HSRP-aware PIM ties them together: it tunes PIM DR priority to follow HSRP state (and sources a PIM Hello from the HSRP VIP), so the active router is always the DR and downstream devices can simply point at the VIP. Failover then follows HSRP.

PIM fine print - shared segments and the SPT switchover

A multiaccess segment runs three independent elections, and they need not land on the same router. The IGMP querier (v2: lowest IP) polls hosts; the PIM DR (Hello-elected, highest IP/priority) is the control-plane agent - in sparse mode it registers sources and originates joins for the segment; and when two routers both forward a group onto the LAN, the duplicate triggers the assert election, which picks the actual forwarder by lowest administrative distance, then lowest metric, then highest IP toward the source - meaning the unicast routing design elects the multicast forwarder on shared segments, and IGP tuning steers multicast there whether you meant it to or not. Prunes behave differently on LANs too: a prune is multicast to all PIM routers and the upstream holds a ~3-second override window in which any other router that still wants the traffic races a Join (the prune override) - so one receiver-less router cannot silently starve its neighbours. This whole family is why the vault's HSRP-aware PIM alignment (above) exists: DR, assert winner, querier, and FHRP active are four roles that a design should deliberately co-locate or consciously separate.

The second knob is the RPT->SPT switchover. Sparse mode joins the shared tree first only because a receiver cannot know a source's address until the first packet arrives via the RP; Cisco's default is then to switch to the source tree on the first packet, buying shortest-path delivery at the cost of per-(S,G) state on every router along every source path. Raising the SPT threshold - to a rate, or to infinity - keeps groups on the shared tree: paths stretch through the RP, but state collapses toward *(,G)-only, which is exactly the economy argument that motivates Bidir-PIM** for many-source/many-listener, low-rate applications. The design question is therefore state versus path optimality, decided per deployment - not a default to inherit blindly.

Multicast in an MPLS L3VPN (mVPN)

Customer multicast inside an MPLS L3VPN (MPLS L3VPN and L2VPN) rides multicast distribution trees (MDTs) built in the provider core. The classic draft-rosen model encapsulates customer multicast in GRE and runs PIM in the core: a per-VRF default MDT carries low-rate/control traffic to all PEs of that VPN, and a data MDT is spun up on demand for a high-rate (S,G) so only interested PEs join - sparing the rest. The modern NG-MVPN (BGP-mVPN) model replaces core PIM/GRE with BGP signalling (customer multicast state carried in BGP, like unicast VPNv4) over P2MP LSPs (mLDP or RSVP-TE P2MP), so the core runs no multicast protocol of its own. Design driver: NG-MVPN scales and aligns with an MPLS/BGP core, whereas draft-rosen leans on running PIM in the provider core.

Choosing the mVPN profile - Profile 14 and the BGP overlay

The mVPN space is a menu of roughly 27 profiles (plus three Tree-SID additions) mixing an underlay tree technology (PIM, mLDP, P2MP-TE, ingress replication, controller-computed) with an overlay signaling protocol (PIM or BGP). The design shortlist is short: Profile 0 is the draft-Rosen model above (GRE encapsulation, PIM in the core, an always-on default MDT); P2MP-TE-based profiles exist where per-tree bandwidth control is the requirement; and Profile 14 - "Partitioned MDT", an mLDP P2MP underlay with a BGP overlay - is the recommended default for a new MPLS core.

The case for Profile 14 is a state-and-protocol argument. It removes the default MDT entirely: partitioned MDTs are unidirectional trees built on demand when customer traffic appears, connecting only the subset of PEs involved - nothing idles on every P router for every VPN, and the model is optimized for sources co-located in a few sites. It also supports anycast sources cleanly: two roots may forward the same (S,G) on separate trees, where a shared default MDT would trigger PIM asserts. mLDP itself is deliberately simpler than core PIM: receiver-driven, downstream-only label signaling from each egress PE toward the tree root (the ingress PE, found by the RPF lookup toward the source), riding the existing reliable LDP sessions instead of PIM's periodic soft-state refresh. A tree is identified by exactly three values - P2MP type, root, and an opaque ID assigned by the root - and a P router holds one mLDP database entry and one LFIB entry per tree, never interpreting the opaque value. The data plane is plain MPLS with replication at the branch points: QoS via EXP propagation, normal TTL behaviour, MPLS OAM for troubleshooting, one label in steady state and two labels under protection.

Protection and the SR future both lean on the unicast toolset. mLDP fast reroute uses precomputed LFA/TI-LFA per-prefix backups - link protection only, no node protection - or RSVP-TE backup auto-tunnels where TE already exists (Fast convergence techniques). And Profile 14 is the SR-coexistence answer: an SR-MPLS core retires LDP for unicast, but SR has no native multicast replication, so either keep (m)LDP alive for multicast alone - mLDP-only Session Advertisement Control (RFC 7473) negotiates away the unicast label exchange at LDP session establishment - or move to Tree-SID, where an SR-PCE controller computes P2MP trees from SR MPLS labels (statically or dynamically, the overlay still BGP mVPN). The trade is a controller dependency on the multicast plane versus keeping a second label protocol (Segment Routing).

The BGP overlay gives one address family two jobs. Auto-discovery announces which PEs participate and binds the core tree via the PMSI Tunnel Attribute, which names the tree technology (PIM, mLDP, P2MP-TE, ingress replication) - so overlay signaling is decoupled from the underlay tree choice. Customer multicast signaling then replaces PIM between PEs: a shared-tree (*,G) or source-tree (S,G) join is a BGP route whose update means join and whose withdraw means prune - PIM survives only on the PE-CE edge. An elegant byproduct: a Source Active route lets BGP replace MSDP for anycast-RP inside the VPN - put the anycast RP address on every PE (all PEs are RPs), and the ingress PE's source announcement reaches them all (RFC 6513) - a third member of the anycast-RP family above. Auto-RP/BSR discovery needs no extra design either: PEs automatically build an additional P2MP control tree for the discovery groups when they see RP information. Two pieces of fine print: SSM inside the VPN is by far the simplest signaling (a single source-tree-join chain end to end), while sparse mode choreographs registering, shared-to-source switchover and source-active announcements across BGP - one more argument for the SSM-where-possible stance; and inter-AS mVPN requires the tree root (the ingress PE loopback) to be reachable in every transit AS's IGP/LDP - redistribute the loopbacks or use recursive FEC. The same machinery also runs without VRFs as global-table multicast (RFC 7716) where a provider wants BGP-signaled, label-switched multicast outside any VPN.

Choosing the core tree - PIM, mLDP, P2MP TE, or ingress replication

The profile menu above hides a cleaner two-axis decision: pick the core tree technology and the overlay signaling independently - the PMSI attribute is what lets them decouple. On the tree axis there are four candidates, and the fourth is not a tree at all. Ingress replication (IR) reuses the existing unicast LSPs: the ingress PE sends one copy per interested egress PE, an extra mVPN label on the stack letting the egress tell multicast from unicast arriving on the same LSP (BGP auto-discovery carries that label). The core holds no multicast state whatsoever and no new protocol appears - protection is simply whatever unicast already has - but the price is paid in ingress and link bandwidth, multiplied by the egress-PE count. That makes IR the honest choice for interop islands (devices that speak neither mLDP nor P2MP TE), low-rate flows, and inter-AS links, and a poor one for high-rate distribution - the WAN cousin of the headend-replication decision under Replication placement below. The other three replicate on the P routers and differ on state, protection and steering: PIM in the core is the lowest common denominator (runs anywhere, GRE, but soft state and no FRR - restoration is IGP reconvergence); mLDP is the balanced default (hard state, LFA/TE protection, both P2MP and MP2MP shapes, follows unicast routing - best for many-to-many); P2MP TE is the only one with explicit paths and RSVP bandwidth reservation plus inherent link/node FRR, at the cost of the highest midpoint state and RSVP soft-state refresh - the few-to-many engineered video tool, tail-ends either configured or discovered via BGP. Selection is requirement-driven, not ideological - and everything is per VPN, so one core can run Rosen for a legacy VPN beside partitioned mLDP for the rest: bandwidth guarantees -> P2MP TE; oldest platforms in path -> PIM/GRE; low traffic or a no-new-core-protocol mandate -> IR; otherwise mLDP, with the overlay chosen next.

MP2MP trees and protecting the root

mLDP's two shapes divide the labour. A P2MP tree is receiver-driven with the root learned from routing (the ingress PE - its BGP next-hop), which is why data and partitioned MDTs need no per-tree configuration. An MP2MP tree has a configured root (any P or PE) and exists for one job: the default MDT - one any-to-any tree replaces a full mesh of per-PE P2MP trees, upstream traffic flowing toward the root and back down, while the data plane still replicates as P2MP along it. A configured root is a single point of failure, so Root Node Redundancy comes in two familiar flavours: anycast RNR puts one root address on two nodes (longest-match, phantom-style - the address need not sit on a real interface): one tree, failover at IGP convergence speed. Hot-standby RNR gives the roots distinct addresses and keeps both trees permanently built: each PE transmits onto its preferred root's tree but accepts from either, buying near-hitless root failover for double the tree state. It is the phantom-RP / anycast-RP decision transplanted into the label core - the same clock-versus-state reasoning.

Overlay signaling - PIM, BGP, in-band, or static

With the tree chosen, the C-multicast overlay has four options. PIM in the overlay keeps the familiar machinery: PEs become one-hop PIM neighbours across the default (or partitioned) MDT and joins/prunes flow to the specific upstream PE - zero new procedures, but a full mesh of PE adjacencies with periodic soft-state refresh, which caps the comfortable PE count. BGP in the overlay (the NG-mVPN doctrine above) turns joins into hard state through the route reflectors: adjacency count collapses to the RR set and updates are event-driven - validated at the ten-thousand-PE mark (railway-CCTV-class designs) where a PIM full mesh is unthinkable - at the price of new procedures (the sparse-mode choreography) and of c-routes fanning out to all PEs via the RRs rather than only the one that needs them. In-band signaling removes the overlay entirely: the egress PE translates the PIM join into the mLDP FEC's opaque value (and the ingress translates back), so the C-(S,G) itself builds the core tree - nothing extra to operate, but every customer flow becomes a tree in the core, the exact opposite of aggregation - right for a bounded set of SSM transit feeds (broadcast contribution), wrong for a general enterprise VPN. Static survives only on P2MP TE: the head-end maps configured groups onto an engineered tunnel (tail-end RPF pointing back), aggregating several C-flows onto one reserved-bandwidth tree - the TV-distribution special.

Migrating an mVPN - co-existence, not cutover

Core tree and overlay migrate independently, and each migrates the same way: run both, choose per egress. Enable the new core tree beside the old; the ingress PE joins both MDTs and transmits onto both, and each egress PE selects by routing policy - per group, per source, or per ingress PE - which tree (and which overlay) it consumes. Migration then proceeds egress-by-egress or application-by-application with instant rollback (flip the policy back), and only when the last egress has moved is the old MDT - and finally the old core protocol - removed. Two operational teeth: the ingress must stay attached to both core trees for the duration, and data-MDT signaling must remain visible in both worlds (the PIM Data-MDT join TLV still has to be sent for the PIM-side listeners). The same lever scales up: global-table multicast lets a new mLDP island grow inside a legacy PIM core - a new inner core displacing the old gradually - reusing the mVPN procedures without VRFs. This is the standard parallel-plane migration shape (Reference models and migration considerations) applied to the multicast plane.

Root reachability across domains - inter-AS, CsC, and seamless MPLS

Every label-switched tree is built toward a root the local RIB must resolve - and three legitimate architectures hide exactly that route. Inter-AS Option A needs nothing (per-VRF back-to-back: each AS runs its own mVPN, any model, independently), but Options B and C stop carrying remote-PE loopbacks in the IGP, CsC hides the customer carrier's internals from the backbone's P routers, and seamless/unified MPLS does it inside one AS by design - BGP-LU carries the PE loopbacks, so a remote-area P router cannot RPF or root toward them (MPLS L3VPN and L2VPN). The fixes are protocol-symmetric. A PIM core uses the PIM Vector: the join carries a proxy (the ASBR/ABR) and routers RPF toward the vector until the domain where the real root is native. An mLDP core uses the recursive FEC: the tree is rooted at the border node, with the true root carried inside an embedded FEC the border unwraps and continues. At larger scale the tree itself can be segmented - per-domain segments of possibly different tree technologies, stitched at the A(S)BRs, which regenerate the mVPN routes - so each domain picks the tree type its hardware and policy prefer.

Multicast over multipoint overlays (NBMA)

An mGRE/DMVPN cloud is one logical interface fronting many neighbours, and default PIM state is kept per outgoing interface - so when a single spoke leaves a group, the prune removes the interface from the OIL and every spoke loses the stream. PIM NBMA mode fixes the granularity: join/prune state is tracked per neighbour address, treating the cloud as a bundle of point-to-points (sparse-mode only). The capacity corollary stands regardless: the hub replicates one copy per joined spoke onto the same physical uplink, so hub bandwidth - not group count - bounds multicast over DMVPN; if replication-in-the-WAN is the real requirement, that is GETVPN-over-private-core territory (Tunneling technology selection).

Securing the multicast control plane

Multicast's default posture is open - any host can send to any group, and any host can join - so the security work is mostly control-plane containment, layered from the edge inward (Securing routing protocols covers the unicast discipline this builds on). At the domain edge, a global group-range permit-list plus multicast boundaries ensure denied groups never create state anywhere - PIM, IGMP/MLD, MSDP and the data plane all drop them, so containment works by absence of state rather than per-router filtering. The RP-distribution choice carries a security asymmetry: Auto-RP rides UDP to flooded well-known groups (224.0.1.39/.40), so its announce/discovery messages must be explicitly filtered at every boundary (rogue candidate-RPs and mapping agents are a real attack), while BSR propagates hop-by-hop link-local and border containment is inherent - one more argument for BSR or static/anycast RP toward inter-domain edges. Protect the RP itself: an accept-register filter drops registers from unauthorised sources at the control plane (the rogue-source defence), and register rate-limiting keeps a misbehaving first hop from exhausting RP CPU. On host segments, a PIM neighbour filter stops end systems posing as PIM routers - the shared-segment elections above are attack surface, not just operations. Receiver-side, IGMP/MLD filters bound which groups a port may join, and state limits (IGMP cache caps, per-interface mroute caps) turn state-exhaustion attacks into bounded local failures. MSDP is the inter-domain trust boundary: filter SA content (accept (S,G) only, scoped to expected groups), cap SA state, and authenticate peers - the same edge discipline as eBGP. One CoPP fine point: the first packet of a PIM-SM flow is effectively control-plane (it creates the state), so CoPP/rACL policies must permit - rate-limited - multicast control including the initial data packets of allowed groups, or the design blackholes itself (Infrastructure hardening - control, management, and data plane).

Replication placement

A multicast design also decides where packets get copied. Replication happens at tree branch points, so the branch points are the design lever: replicate at the source/headend (per-receiver unicast copies - simplest, but ingress bandwidth scales with audience size), at the network edge, or inside the network as close to the receivers as possible - the efficient default, because each link then carries one copy regardless of receiver count. On shared segments, let Layer 2 do the copying via IGMP/MLD snooping (Layer 2 multicast - IGMP and MLD snooping) rather than pushing per-receiver copies from upstream. The same choice returns in overlays as ingress (headend) replication vs an underlay multicast tree for BUM traffic (Overlays and BGP EVPN): headend replication avoids running PIM in the underlay but costs ingress bandwidth per remote VTEP; an underlay tree is efficient at scale but is a second control plane to operate.

Recommendation / justification

Use SSM where sources are known (no RP, simplest, most robust); Bidir for many-to-many; otherwise PIM-SM with Anycast-RP for resilience. Justify by the source/receiver model and the need for RP high availability.

What would change this

• Sources are well-known and few -> SSM (drop the RP entirely).
• Huge number of sources, many-to-many -> Bidir-PIM.
• Need RP HA -> Anycast-RP (MSDP).

IPv6 / dual-stack note

PIM runs for IPv6 too; SSM uses MLDv2; IPv6 supports embedded-RP (RP address encoded in the group), simplifying RP distribution. In mVPN, the core trees themselves are IPv4-only - PIM, mLDP and P2MP TE all build v4 trees - so IPv6 customer multicast rides the same IPv4-signalled trees, an IPv6 explicit-null label at the bottom of the stack demuxing v4 from v6 on a shared MDT; only the overlay is dual-stack (PIMv6 or BGP IPv6-mVPN).

Related

• Network hierarchy and topologies
• IGP selection - OSPF vs IS-IS vs EIGRP

Spaced repetition

Sources

• Cisco Live BRKIPM-3017 (delivered as the CLUS 2019 deck, L. De Ghein): core-tree and overlay comparisons incl. IR and in-band, MP2MP root redundancy, MoFRR, migration co-existence doctrine, PIM Vector and seamless-MPLS root reachability, IPv6 explicit-null demux, profile decision logic.
• Cisco Live BRKENT-2004 (CLUS 2022, L. De Ghein): Profile 14 / partitioned MDT doctrine, mLDP mechanics and FRR limits, BGP mVPN overlay and RT-5 anycast-RP, mLDP-only SAC, Tree-SID, RFC 7716.
• Cisco Press, Designing for Cisco Network Service Architectures (ARCH), 4th Ed. - multicast control-plane security: boundaries, Auto-RP/BSR asymmetry, accept-register, MSDP filtering, CoPP nuance.
• Cisco Press, CCIE Routing and Switching v5.0 OCG Vol 2, 5th Ed. - prune override, assert election, DR roles, SPT switchover.
• Cisco Live BRKRST-2041 (2013): PIM NBMA mode, per-spoke hub replication bound.
• Cisco Press, IP Multicast Vol I-II; Routing TCP/IP Vol II.

domain: Core · blueprint-ref: Core 3.9 Multicast routing concepts · type: design-decision · status: complete · tags: