SD-WAN design
Requirement / business driver
A transport-independent, app-aware, centrally-managed, secure WAN - decoupling policy from the underlying circuits.
The four planes
- Orchestration - zero-touch onboarding and certificate/trust bring-up.
- Management - central controller for config, monitoring, policy.
- Control - distributes routes and policy to edges (an OMP-style overlay control plane).
- Data - IPsec overlay tunnels across any transport (MPLS, internet, LTE/5G).
Cisco SD-WAN components (terminology)
Mapping the generic planes to Cisco's SD-WAN (Viptela) realisation: vBond (orchestration - authenticates edges, NAT traversal), vManage (management - GUI/config/monitoring), vSmart (control - runs OMP, distributes routes and policy), and cEdge/vEdge (data plane - the WAN edge routers). OMP advertises OMP routes, TLOC routes, and service routes over the overlay. A TLOC (transport locator) identifies an edge's attachment to a transport (system-IP + colour + encapsulation); a TLOC extension lets one edge reach a transport through a co-located edge when it has no direct circuit.
Control-plane mechanics - trust, keys, and best path
The vSmart behaves like a BGP route reflector: it receives every edge's routes and TLOCs, applies policy, and reflects the result - and every persona pair (vBond, vManage, vSmart, edges) authenticates by certificate and talks over DTLS/TLS control channels, while edges hold no control sessions with each other at all, only IPsec data tunnels - so control state scales with controllers, not with site count. The data plane is IKE-less: each edge's IPsec keys are advertised through OMP via vSmart, collapsing the classic n-squared key-exchange problem, with pairwise keying as the opt-in for unique per-peer-pair keys. Controllers are cloud-hosted in the common model, or on-premises where sovereignty or air-gap requirements demand it.
OMP best path, once a route's TLOC is valid (validity means an active BFD session to that TLOC - the overlay's next-hop-reachable test): locally sourced beats learned -> lower administrative distance -> higher OMP preference -> higher TLOC preference -> origin ladder (connected, static, eBGP, EIGRP internal, OSPF intra -> inter -> external, EIGRP external, iBGP) -> lower origin metric -> highest system IP; ECMP is 4 routes by default, up to 16. Brownfield loop prevention where OMP and the site IGP/BGP are mutually redistributed at multi-exit sites uses a per-protocol trick: OSPF's down bit, a BGP Site-of-Origin community carrying the site ID, and for EIGRP an external-protocol marking of OMP-Agent installed at AD 252 so OMP's 251 stays preferred.
Which tunnels form - colours, restrict, and NAT
A colour is the transport tag on a TLOC, and its category is a NAT statement: private colours assume no NAT (edges connect using underlay addresses), public colours are NAT-safe (the mapping is discovered during orchestration). Default behaviour is aggressive: edges attempt full-mesh tunnels across every colour pair with IP reachability - so MPLS-attached routers will patiently try to reach internet colours they can never touch unless told otherwise. Restrict pins a colour to same-colour tunnels only; tunnel groups generalise the idea to named sets. NAT type then decides feasibility: symmetric NAT accepts no unsolicited inbound, so two edges both behind symmetric NAT cannot build a direct tunnel - one side needs full-cone behaviour, or their traffic transits a mutually reachable site - the quiet reason home-office and LTE sites sometimes refuse spoke-to-spoke.
Data-plane and OMP fine print from the CVD
Three refinements survive against this note's baseline. First, QoS after encryption breaks anti-replay: low-priority packets queued post-encryption can fall outside the receiver's replay window, so IOS XE runs multiple sequence-number spaces (multi-SNS) - one space per egress queue, encoded in the SPI field - which removes queue-induced reordering from the replay check entirely (both tunnel ends should carry matching class counts, and the window itself is worth raising from 512 to the 4096 maximum). Second, control-plane loss is survivable by design but the timers are coupled: OMP graceful restart defaults to 12 hours of forwarding on last-known state, and because controllers distribute the IPsec keys, the rekey interval must stay at least twice the GR timer or keys age out mid-outage. Third, path diversity is a controller setting, not a default: OMP advertises best paths only and four per prefix - enable send-backup-paths and raise send-path-limit toward 16 on the controllers (and the edge's ecmp-limit if more installed paths are wanted), and for overlays past ~500 devices stretch the OMP hold timer to 300 s. Pairwise keys are the optional per-peer-key hardening over the default group-style TLOC keys. On the migration side, the parallel-fabric interconnect gains one more loop-prevention lever alongside same-ASN and tags - overlay-AS, stamping OMP-sourced routes with a shared AS number every edge recognises and refuses to relearn.
Policy and segmentation
Per-segment VPNs for tenants; app-aware routing with SLA probes and automatic remediation (FEC, duplication, path switch); topologies (hub-spoke, mesh, regional hubs); integrated security / SASE (NGFW, SWG, ZTNA) at the edge or cloud.
Policy - where each type is enforced
The policy families differ most in where they execute. Centralized control ("topology") policy and VPN membership policy are enforced on the vSmarts themselves - nothing is pushed to the edges; a site simply receives an already-filtered view of the fabric. That is how hub-and-spoke, regional meshes, per-VPN topologies (corporate full mesh beside a PCI hub-and-spoke), extranets to shared services, and DC-transit designs are all carved from one full-mesh-capable fabric. Centralized data policy and app-route policy are encoded into OMP updates toward the edges: data policy is the fabric's policy-based routing - per-application DIA (guest breakout at the branch, trusted-app direct cloud access) with the security consequence that drives edge or cloud-delivered inspection (Perimeter security - firewalls and IPS-IDS), service insertion (an edge advertises a firewall/IPS/proxy as a labelled service route, and policy steers matching inter-site flows through it), FEC and packet duplication. Localized policy stays on the device: route policy is the only local control-plane knob (filter and tag at the OMP <-> BGP/IGP redistribution edge), plus interface QoS and ACLs (QoS design - models and strategy). The edge's forwarding order matters: routing lookup -> ingress ACL -> app-route, which can only choose among equal-cost overlay paths -> data policy, which can override the routing decision outright. And the failure mode is graceful: losing vSmart does not stop forwarding - edges keep the last learned state and relearn policy through OMP on return.
App-route math - the SLA reaction window
BFD does double duty on every tunnel. Liveness: hello interval (default 1 s, negotiated up to the slower peer's value) times the multiplier (default 7) is the dead-tunnel detection time - interface-down skips the wait. Quality: each app-route poll interval is a bucket in which loss, latency, and jitter are computed from those same probes, and SLA compliance is judged over the last app-route-multiplier buckets (default 6) - so the quality reaction window is roughly poll interval x buckets considered, tunable per colour (slower probes on metered LTE). The SLA class action then walks a ladder: compliant preferred colours load-share per flow -> a single compliant preferred carries it alone -> any compliant non-preferred colour -> and when nothing measures compliant, either backup-sla-preferred-colour pins traffic to the contractual circuit anyway or strict drops it rather than violate. With no preferred colour at all, traffic load-shares across every compliant colour.
Cloud OnRamp (SaaS and IaaS)
SD-WAN extends app-aware policy into the cloud via Cloud OnRamp. OnRamp for SaaS continuously probes multiple paths to a SaaS application (direct internet at the branch, via a regional gateway, via the DC) and steers each site onto the best-performing path by loss/latency - so SaaS quality is measured and chosen, not assumed. OnRamp for IaaS / multicloud automates branch-to-cloud connectivity by instantiating SD-WAN edges (cloud gateways) inside the cloud provider, extending the overlay and its segmentation/policy into the VPC/VNet instead of hand-building IPsec and routing. This is SD-WAN's answer to the cloud-connectivity design question (WAN transport architecture selection) alongside direct-connect and colo options.
Integrating and migrating - DC first, branch patterns, underlay ladder
Migration doctrine starts with archaeology - topology, HA, traffic engineering, and feature parity of the legacy WAN - then migrates the data centres first so they act as transit hubs between migrated and unmigrated sites; size DC WAN bandwidth for that hairpin era, and regionalise hubs per geography when transit latency bites. The transport side stays deliberately dumb: one VPN 0 interface per transport with a default route to the carrier edge (prefer 1:1 static NAT where internet NAT is unavoidable), and every DC edge needs reachability into every transport - directly or via TLOC extension - or HA and app-aware routing degrade. Loopback TLOCs (bound to a physical interface) decouple the TLOC from any single port for indirect termination or underlay-integrated designs. Branch patterns: single-edge CE replacement takes downtime (L2 service side auto-originates connected routes into OMP; L3 needs explicit mutual OMP <-> IGP redistribution); dual-edge allows near-hitless cutover - shift traffic with VRRP priority or routing metrics beside the legacy CE, convert one router at a time; and a single circuit handoff shared by two edges is a TLOC extension - a private colour needs a routable transit network advertised into VPN 0 toward the provider, a public colour NATs on the extending router, and IOS-XE adds a GRE L3 variant for non-adjacent pairs. Underlay integration is a ladder: overlay-only -> underlay as backup -> full integration (overlay to migrated sites, underlay to unmigrated). The last rung adds real routing complexity - it wants iBGP's anti-transit behaviour (or disciplined tag-and-filter) and overlay-favouring metrics to keep symmetry - and is rarely justified: voice tolerates roughly 300 ms round trip, so hub transit usually suffices during coexistence.
Quality of Experience - the user-side contract
The blueprint's user-and-application-experience row has a name the SD-WAN literature uses: Quality of Experience (QoE) - a superset of QoS. QoS schedules packets; QoE composes everything the session experiences: access and security posture, application-aware routing and dynamic path choice, segmentation, and end-to-end visibility, applied as one policy. The design mechanism that delivers it is identity anchoring: because policy lives in the overlay and is bound to the user or session, not the port, subnet, or VLAN, the contract travels with the user - move a worker from a branch to another site or onto a home broadband circuit and the same experience policy re-attaches wherever they land. The lens for scenarios: when a requirement says "consistent user experience anywhere", the answer is not a QoS class - it is overlay policy anchored to identity, with QoS as one ingredient.
Recommendation / justification
Choose SD-WAN when app-aware path control, multi-transport, agility (ZTP), central policy, and integrated security matter - which is most modern WAN refreshes. See the head-to-head in WAN transport architecture selection.
What would change this (mid-scenario twist)
- Strict, contractual SLA on some traffic -> keep MPLS for it (hybrid under SD-WAN).
- Pure cost play, Cisco shop, no app-awareness needed -> DMVPN may suffice.
Validation checks
- Are critical apps mapped to SLA classes with measurable remediation?
- Is the control/orchestration plane redundant, and onboarding secured (certs)?
IPv6 / dual-stack note
SD-WAN overlays and service VPNs support dual-stack; validate v6 across the underlays and policies.
Related
- WAN transport architecture selection
- Tunneling technology selection
- QoS design - models and strategy
Spaced repetition
Name the four SD-WAN planes.
Orchestration, management, control, data.
SD-WAN's defining capability over a routed overlay is [...].
SD-WAN's defining capability over a routed overlay is app-aware path control with SLA probing and remediation.
In a hybrid SD-WAN, you typically keep MPLS for [...].
In a hybrid SD-WAN, you typically keep MPLS for traffic needing a strict contractual SLA.
Map Cisco SD-WAN components to the planes.
vBond = orchestration, vManage = management, vSmart = control (runs OMP), cEdge/vEdge = data plane; OMP carries OMP/TLOC/service routes, and a TLOC is an edge's transport attachment (system-IP + colour + encap).
What does SD-WAN Cloud OnRamp for SaaS do?
It probes multiple paths to a SaaS app (direct internet, regional gateway, via DC) and steers each site onto the best-performing path by loss/latency - measuring SaaS quality rather than assuming it.
How does Cloud OnRamp for IaaS/multicloud connect branches to the cloud?
It instantiates SD-WAN edges (cloud gateways) inside the cloud provider, extending the overlay's segmentation and policy into the VPC/VNet automatically, instead of hand-building IPsec tunnels and routing.
OMP loop prevention when redistributing at dual-exit sites?
OSPF down bit, a BGP Site-of-Origin community carrying the site ID, and EIGRP external-protocol OMP-Agent installed at AD 252 so OMP (AD 251) stays preferred.
By default WAN Edges attempt [...] - restrict and tunnel groups pin colours, and [...] blocks a direct spoke-to-spoke tunnel.
By default WAN Edges attempt full-mesh tunnels across every colour pair with IP reachability - restrict and tunnel groups pin colours, and symmetric NAT on both ends blocks a direct spoke-to-spoke tunnel.
Where is centralized control (topology) policy enforced, and what does an edge see?
On the vSmart controllers themselves - never pushed to edges; each site just receives an already-filtered routing view, which is how topologies, extranets, and per-VPN designs are built.
SD-WAN SLA reaction math?
Liveness = hello interval x multiplier (defaults 1 s x 7, hello negotiated to the slower peer); quality = per-poll-interval buckets of loss/latency/jitter judged over the app-route multiplier's buckets (default 6) - reaction window ~ poll interval x buckets.
App-route policy can only choose among [...], while a centralized data policy can [...].
App-route policy can only choose among equal-cost routes from the routing lookup, while a centralized data policy can override the routing decision entirely.
Dual-edge branch, one circuit handoff - how do both routers use it?
TLOC extension: private colour = routable transit advertised into VPN 0 toward the SP; public colour = NAT on the extending router; IOS-XE offers a GRE L3 variant for non-adjacent routers.
Why does SD-WAN run multiple sequence-number spaces (multi-SNS)?
QoS queuing happens after encryption, so delayed low-priority packets can miss the anti-replay window; multi-SNS gives each egress queue its own sequence space (encoded in the SPI), eliminating queue-induced replay drops - pair it with the 4096 replay window, and keep IPsec rekey at least twice the 12-hour OMP graceful-restart timer.
QoE vs QoS in SD-WAN design?
QoS schedules packets; QoE is the superset - access, security, app-aware routing, path choice, segmentation, and visibility composed as one overlay policy anchored to user or session identity, so the experience contract follows the user across sites and transports.
Sources
- Z. Zsiga, Guide to the Benefits of SD-WAN v1.3 (Zigbits, 2020; user-supplied PDF, full pass 2026-07-26) - QoE-vs-QoS and identity-anchored experience portability.
- Cisco, Catalyst SD-WAN Design Guide (CVD) + IWAN to SD-WAN Migration Guide (bonus batch, fetched) - multi-SNS, GR/rekey coupling, OMP path-diversity knobs, overlay-AS.
- Cisco Press, Cisco Software-Defined Wide Area Networks - full pass (batch 20): OMP/colour mechanics, policy architecture, app-route SLA math, DC/branch integration and migration.