Tunneling technology selection

Requirement / business driver

Pick the site-to-site / WAN overlay that meets the customer's needs for encryption, scale, dynamic connectivity, multicast/routing, and transport (private vs internet).

Constraints

• Encryption mandated? Transport = internet or private MPLS/L2?
• Spoke count and need for dynamic spoke-to-spoke mesh?
• Must the overlay carry routing protocols / multicast?
• Operational model and team familiarity.

Options

• DMVPN - mGRE + NHRP + IPsec; hub-and-spoke that builds dynamic spoke-to-spoke tunnels.
• GETVPN - tunnel-less group encryption that preserves routing/multicast; for private (MPLS/L2) any-to-any.
• IPsec (P2P / VTI) - point-to-point encrypted links.
• GRE - simple encapsulation (no encryption alone; pair with IPsec).
• MPLS L3VPN - SP-managed any-to-any (private).

Comparison

Tradeoffs

DMVPN scales internet WANs with dynamic mesh; GETVPN gives any-to-any encryption over a private core while preserving native routing/multicast; IPsec/VTI suits a handful of links; GRE is encapsulation/routing only; MPLS L3VPN is managed any-to-any at scale.

Count the control planes. Every tunnel design runs two routing instances: one that reaches the tunnel endpoints (the underlay/transport - possibly the provider's, not yours) and one for reachability through the overlay. Tunneling therefore displaces complexity across two planes rather than removing it - design, monitor, and budget operations for both, and mind their interaction: endpoint-reachability flaps tear down everything the overlay carries (Factors affecting convergence).

The named failure mode of that interaction is recursive routing: if the transport network or the tunnel endpoint addresses are ever advertised into the overlay's routing protocol - a redistribution slip, an over-wide network statement, or a hub pushing a default route across the tunnel - the router learns the path to the tunnel endpoint through the tunnel itself. A tunnel cannot encapsulate through itself, so it is torn down, the overlay adjacency drops, the underlay path returns, the tunnel re-forms, and the cycle flaps indefinitely (point-to-point GRE is detected and disabled with a recursive-routing event; multipoint/DMVPN shows a looped-chain variant). The structural fix is the front-door VRF (FVRF): bind the transport interface to its own VRF and source the VRF-aware tunnel from it, so underlay and overlay live in separate routing tables that cannot mix - recursion becomes impossible by construction rather than by filtering discipline. With one FVRF per transport (internet, MPLS, LTE), the outbound interface for each tunnel is also pinned deterministically - the same pattern SD-WAN productizes as transport VRFs (SD-WAN design). The separation also enables the central-Internet design: the ISP-learned default lives in the FVRF purely for tunnel establishment, while the hub advertises the global-table default through the tunnel - branch users reach the Internet via the headquarters security stack with no split tunnelling, and the two defaults can never collide.

DMVPN phases (routing and policy consequences)

DMVPN's phase is a design decision, not a version number. Phase 1 builds hub-and-spoke tunnels only - every spoke-to-spoke flow hairpins through the hub, which is a feature when policy demands that all inter-branch traffic transit a central inspection point (the classic exam reasoning: pick Phase 1 / hub-routed precisely because everything must hit the central firewall). Phase 3 uses NHRP redirects/shortcuts to build dynamic spoke-to-spoke tunnels on demand and - unlike the legacy Phase 2 - lets the hub summarize toward the spokes, so it wins on scale, hub load, and spoke-to-spoke latency (voice/video between branches). The selection question is therefore: must inter-branch traffic transit a central policy point? Yes -> Phase 1 (or hub-routed by design); no -> Phase 3.

Inside the phases - the next-hop rule

One mechanism generates every difference between the phases: what the hub does to the IP next-hop when it re-advertises spoke routes. In Phases 1 and 3 the hub rewrites the next-hop to itself (EIGRP/BGP next-hop-self behaviour; OSPF runs the point-to-multipoint network type), so spokes are content with a summary or a default pointing at the hub, spoke routing tables stay small, hub count is unconstrained, and the hubs may even run a different routing protocol among themselves than toward the spokes. Phase 2 must preserve the originating spoke's tunnel address as the next-hop - that specific next-hop is what triggers its spoke-initiated direct tunnels - and everything expensive follows from that one requirement: the hub cannot summarise (each spoke needs every remote specific with next-hop intact), OSPF is forced onto the broadcast network type whose DR/BDR mechanics practically cap the design at two hubs, multi-hub designs must daisy-chain, and hub-to-hub must run the same protocol as hub-to-spoke. Phase 3 breaks the coupling by moving the trigger out of the routing table into NHRP: traffic first follows the summary through the hub, the hub answers with an NHRP redirect, the spoke resolves the destination and installs a shortcut - an NHRP-sourced /32 next-hop override in the RIB - and the flow takes the direct path; Phase 1's routing simplicity with Phase 2's data-plane directness. Two asymmetries remain to design for: the routing protocol and multicast run only over the hub-and-spoke tunnels - the IGP never sees an on-demand spoke-to-spoke tunnel, so direct-path liveness is NHRP's job (holdtime and registration refresh), not the routing protocol's; and because NHRP resolution walks the routing table, hierarchical designs resolve through the hub tiers (IGP selection - OSPF vs IS-IS vs EIGRP carries the stub/network-type doctrine).

Scaling DMVPN past one flat hub

When one hub's tunnel, routing, or crypto capacity runs out, the ladder is: regional/hierarchical hubs (spokes home to a regional hub and the hubs interconnect - NHRP resolution walks the hierarchy) -> load-balanced hub clusters (several hubs behind one address share the spoke population) -> and, when the real pressure is segmentation rather than spoke count, MPLS over DMVPN (2547oDMVPN): run the L3VPN label machinery over one DMVPN cloud, so many VRFs multiplex a single overlay instead of building a tunnel cloud per VRF (MPLS L3VPN and L2VPN) - the self-run equivalent of the provider's segmented WAN. One service-insertion trap on the hub rides with Phase 3: traffic interception configured on the mGRE tunnel itself (WCCP-class redirection) consumes packets before the NHRP redirect can fire, so spoke-to-spoke shortcuts never form and everything hairpins - place the interception on the hub's LAN-side interface instead, letting the first hairpinned packets trigger the redirect while later ones ride the shortcut.

IPsec on the overlay - mode, liveness, and replay

Three IPsec decisions ride along with any encrypted overlay. Mode: GRE already provides the tunnel, so DMVPN encrypts in transport mode - ESP wraps the GRE payload without adding a second outer IP header, saving ~20 bytes of the MTU budget (MTU, fragmentation, and PMTUD - the classic anchors on a 1500-byte transport: ip mtu 1400 on the tunnel, TCP MSS clamped to 1360), and transport mode is also what keeps NHRP working through NAT; tunnel mode earns its extra header only when the crypto endpoints differ from the tunnel endpoints. (GRE itself costs 24 bytes - 20 IP + 4 flags - plus 4 more if a tunnel key is set.) Liveness: IPsec does not natively notice a dead peer - traffic blackholes until the SA lifetime expires - so dead peer detection (DPD) probes on demand when liveness is in doubt. The timer rule is layered failure detection: size DPD above the routing protocol's hold time, so routing convergence fires first and DPD is the backstop for the case where the routing relationship itself is broken; run it on the spokes, not the hub, because per-peer probe state scales with branch count. Behind NAT, NAT keepalives hold the translation open for spoke-to-spoke tunnels, which carry no routing-protocol traffic to do it for them. Replay vs QoS: ESP stamps sequence numbers at encryption, before QoS queueing - priority scheduling then reorders the wire, and delayed low-priority packets arrive behind the receiver's anti-replay window (default 64) and are dropped as replays. The QoS design worked; the crypto ate the packets. Enlarge the replay window (up to 1024; ~128 bytes per SA, no throughput cost) whenever queueing rides an encrypted SA (QoS design - models and strategy).

Registration is its own control plane to protect. Spokes re-register at one third of the NHRP holdtime, and a spoke whose transport address has changed (DHCP renewal, dynamic NAT) presents the hub with a conflict against its own stale registration - by default the hub rejects the newcomer until the old entry ages out, blackholing that branch for up to the holdtime. Dynamically addressed estates therefore register non-unique and keep holdtimes at minutes rather than the multi-hour default; the hub itself needs a fixed, statically translated NBMA address - dynamic NAT is a spoke-side liberty only. When a hub or transport event makes the whole spoke population re-register and re-key at once, IKE call admission control caps concurrent negotiations so the crypto plane degrades a queue at a time instead of collapsing. Certificate-authenticated overlays add one more hidden dependency: IKE validates the peer's chain against the revocation infrastructure, so an unreachable CRL/OCSP point quietly fails every new tunnel establishment and re-authentication while established SAs coast toward expiry - give revocation distribution the same availability engineering as the hubs, or choose its fail-open/fail-closed posture deliberately (AAA and identity - RADIUS, TACACS+, 802.1X).

GETVPN operational fine print - keys, COOP, and policy discipline

The group model earns its scale through machinery worth knowing at design depth. The key server splits duties across a KEK (protecting rekey control messages) and TEKs (the data-plane group SAs). Rekeys come in two shapes: unicast rekey is acknowledged - a GM that misses three consecutive rekeys is removed and must reregister - while multicast rekey scales flat but is unacknowledged and needs a multicast-capable core; either way the rekey should source from a loopback, and a GM independently reregisters about a minute before SA expiry, walking its ordered key-server list. Size the GM platform for roughly twice the steady-state SA count, because rekeys are make-before-break.

COOP turns key servers into a fault-tolerant set: priority elects a primary, there is no preemption (a recovered higher-priority KS stays secondary), and the doctrine is at least two servers, rarely value beyond four, geographically dispersed, placed in parallel with the GMs rather than behind them, all carrying identical policy and the same exportable RSA keypair. Change policy on secondaries first, near TEK end-of-life; and give the KSs a dedicated inter-KS path so a partition cannot mint multiple primaries - remembering that the path must also carry rekeys to be worth anything. Registration load spreads via per-GM ordered server lists or an anycast address. Because one group SA is shared by everyone, per-peer counter anti-replay is meaningless - TBAR substitutes key-server pseudo-time without any NTP dependency. The policy ACL is equally group-shaped: the KS pushes the same interesting-traffic ACL to every GM, so write it as summarised, symmetric permits (order of a hundred permits, since each becomes an SA pair), or invert it - permit-any-any plus explicit control-plane denies; each GM may add a local deny-only ACL that is checked first. The failure posture is a genuine design decision: GETVPN fails open by default (crypto gone, traffic flows clear) - fail-closed requires deliberately installing the inverse deny ACL.

QoS on encrypted overlays

Encryption changes what the classifier can see. By default the ToS byte is copied to the outer header at encapsulation, so pure DSCP classification survives IPsec and GRE with no help; the pre-classify feature exists for everything else - matching on addresses, ports, or flags fails after encryption because crypto runs before classification, and pre-classify keeps a cleartext copy of the inner header for the egress decision on that same router. On DMVPN hubs the problem is aggregation: every spoke terminates on one mGRE interface, so a single flat egress policy lets busy spokes starve quiet ones and lets the hub overrun a small spoke's downlink. Per-tunnel QoS fixes both - the spoke signals an NHRP group at registration and the hub instantiates the mapped per-spoke parent shaper (to that spoke's rate) with the class model as the child, automatically per tunnel. Its limit is architectural: the feature is hub-only, so Phase-3 spoke-to-spoke tunnels bypass it - direct inter-branch traffic runs unshaped unless spokes carry their own policies. GETVPN avoids the whole visibility problem by construction: the original IP header is preserved outside the ESP payload, so PE routing and QoS classification keep working untouched - on a private core, tunnel-mode address hiding buys nothing and only costs transparency, which is precisely why GETVPN is the private-WAN answer (QoS design - models and strategy).

The label boundary - matching virtualisation to the WAN handoff

Which segmentation stack you can run over the WAN is decided by one fact about the handoff: can MPLS labels cross it? Over an SP-managed IP-VPN service the answer is no - the CE peers with the provider's PE in plain IP (eBGP/static) and no labels are exchanged - so self-run segmentation must ride an overlay on top: MPLS VPN over point-to-point GRE (RFC 4023: the top forwarding label is replaced by a GRE/IP header to the far customer-PE, the VPN label is preserved inside, and PE-to-PE behaves as implicit-null) for a few sites, or MPLS VPN over mGRE with BGP-derived tunnel endpoints at scale - the standard RFC 4364 control plane runs unchanged, but the next hop resolves to an IP-encapsulation endpoint learned from BGP itself, so there is no LDP, no NHRP, no per-peer tunnel configuration, one transport address per router, and dynamic any-to-any - a distinct pattern from DMVPN even though both use mGRE. Over an L2 service (E-LINE/E-LAN) labels can cross, because the customer owns both ends of the Ethernet: run native MPLS across it, or take the label-free rung - VRF-Lite with an 802.1Q subinterface per VRF (Infrastructure segmentation), whose tax is explicit: a tag and an IGP instance per VRF per site, honest for a handful of VRFs and ruinous past that - which is precisely the pressure that justifies stepping up to labels. And over a self-deployed IP/MPLS backbone, RFC 4364 is simply native (MPLS L3VPN and L2VPN) - maximum service richness (TE/FRR, L2VPN, per-VRF multicast) at maximum operational ownership. State the handoff first; the stack follows.

Securing the virtualised fabric composes rather than complicates: on an MPLS-VPN-over-mGRE fabric, GETVPN encrypts the mGRE payload - key servers push group keys and policy to every customer-PE as a group member, so the VPNv4 traffic of every VRF is encrypted without any per-peer tunnels, and the any-to-any and multicast behaviour of the fabric survives untouched. It is the same GETVPN logic as above (header preservation, group keying), applied one layer down - encryption as an attribute of the fabric, not a second overlay to operate.

Recommendation / justification

Map to the dominant constraint: internet + many dynamic spokes -> DMVPN; private WAN needing any-to-any encryption + multicast -> GETVPN; a few secure links -> IPsec/VTI; encapsulation/routing over IP -> GRE (+IPsec); managed any-to-any at scale -> MPLS L3VPN. Note that SD-WAN increasingly supersedes DIY overlays - see WAN transport architecture selection.

What would change this (mid-scenario twist)

• Cost becomes dominant -> internet transport (DMVPN / SD-WAN).
• Compliance requires encryption everywhere on a private core -> GETVPN.
• App-aware routing / agility / central policy required -> SD-WAN.

Validation checks

• Does the overlay carry the required routing/multicast, and is encryption applied where mandated?
• Does the spoke count and mesh requirement fit the chosen model's scaling?

IPv6 / dual-stack note

DMVPN/GRE/IPsec support IPv6 (FlexVPN/IKEv2 is the modern, dual-stack-friendly framework); validate v6 over the overlay.

Related

• WAN transport architecture selection
• SD-WAN design
• QoS design - models and strategy
• MTU, fragmentation, and PMTUD - encapsulation overhead and the tunnel MTU budget

Spaced repetition

Sources

• Cisco Live BRKSEC-3052 (Demystifying DMVPN, R. Jwalaniah; 2018 deck, user-supplied PDF, all 148 pages passed 2026-07-26) - phase next-hop/summarisation doctrine, control-vs-data-plane asymmetry, registration hygiene, CRL dependency, MTU anchors.
• Cisco, GETVPN Design and Implementation Guide v1.0 (bonus batch, fetched; retrieval truncated around section 3.7.4) - KEK/TEK, rekey and COOP doctrine, TBAR, policy and fail-mode discipline.
• Cisco Live BRKRST-2045 (Network Virtualisation Design Concepts over the WAN, 2014, C. Hill) - label-boundary ladder, RFC 4023 MPLSoGRE, MPLS VPN over mGRE (BGP endpoints), GETVPN-over-mGRE.
• Cisco Live BRKRST-2041 (2013): FVRF dual-default/central-Internet framing, hierarchical and 2547oDMVPN scaling; re-passed in full batch 65 (user-supplied PDF, 96 slides): service-insertion/NHRP-redirect placement added.
• Cisco Press, Designing for Cisco Network Service Architectures (ARCH); Cisco VPN design guides.
• Cisco Press, CCNP ENARSI 300-410 OCG (Edgeworth/Lacoste) - recursive routing, FVRF, IPsec modes, DPD, anti-replay.
• Cisco Press, End-to-End QoS Network Design, 2nd Ed. (Szigeti et al.) - pre-classify, per-tunnel QoS, GETVPN header preservation.

domain: Core · blueprint-ref: Core 5.2.a Tunneling technology selection · type: design-decision · status: complete · tags: [core, core/virtualization, tradeoff/scale, tradeoff/security, tradeoff/operability]