Skip to main content

Tenant Address Plan

Every overlay address SecureLink assigns is derived from one number: the tenant's base CIDR. This page documents that derivation, the reservations inside it, the addresses that are deliberately outside it, and a worked example tracing a real packet path hop by hop.

Why this matters

The overlay has no address-resolution protocol of its own. WireGuard allowed_ips is the routing table — a prefix is attached to exactly one peer, and a prefix attached to the wrong peer is silently dropped behind a control plane that looks entirely healthy. Address allocation is therefore a correctness concern, not a tidiness one.

The supernet​

All tenant overlay space comes from a single configurable supernet, carved into one prefix per tenant.

SettingDefaultWhere
default_supernet100.64.0.0/10 (CGNAT, RFC 6598)vsm_system_settings, category networking
default_tenant_prefix16same
auto_allocate_address_spacetruesame

CGNAT space is used deliberately. It is routable-shaped but not globally routed, so it will not collide with a customer's own RFC 1918 estate the way 10.0.0.0/8 reliably does.

Allocation is first-free-/16 ascending, recorded one row per tenant in tenant_address_space. The live allocation is contiguous:

Base CIDRAllocated
100.64.0.0/162026-04-08
100.65.0.0/162026-04-13
100.66.0.0/162026-04-17
100.67.0.0/162026-04-24
100.68.0.0/162026-05-11
100.69.0.0/162026-05-11
100.70.0.0/162026-05-25
100.71.0.0/162026-05-25
100.72.0.0/162026-05-29
100.73.0.0/162026-07-21

A /10 carved into /16s yields 64 tenants. Ten are in use.

Inside a tenant's /16​

The third octet is the service selector. Only three of its values are stored per tenant (connector_octet, ikev2_octet, peering_octet); the rest are conventions in code. All ten live tenants use the defaults 200 / 220 / 240.

Octet 3RangePurposeDerived by
0x.y.0.0/24Ungrouped App VPN clientsgetServiceSubnet(…, 'wg_vpn_ungrouped')
1–199x.y.N.0/24One /24 per Device GroupallocateSubnetForGroup()
200x.y.200.0/24Connector backbone (wg0) — also IoT peersconnector_octet
220x.y.220.0/24IKEv2 App VPN pool, dedicated edgeikev2_octet
221–239x.y.N.0/24IKEv2 per-slot pools, one per MTGE tenant slotikev2_octet + 1 + slot
240x.y.240.0/24E2E peering, primary tunnelpeering_octet
241x.y.241.0/24E2E peering, secondary tunnel (dual-tunnel)peering_octet + 1

Source: vsm-api-v2/src/services/tenantAddressSpaceService.ts.

Host conventions within a /24​

The subnet map alone is not enough to place an address — each range has its own host convention.

App VPN (x.y.0.0/24) — clients get .1 through .253; the server gateway is .254. The wg1 interface carries a /24 mask, not /16, even though Device Group subnets conceptually extend the App VPN space across the whole tenant /16. VPP rejects a /16 here because it would overlap wg0's x.y.200.0/24. Grouped clients are reached by per-peer allowed_ips /32s rather than by a connected route.

Device Groups (x.y.N.0/24) — one /24 per group, hosts .1–.253 allocated in order.

Connector backbone (x.y.200.0/24) — a single /24 shared tenant-wide by three kinds of occupant, all allocated from one pool so they cannot collide:

  • hub edge wg0 gateways, one distinct host per hub (VSNP-1016 connector HA — every hub of the tenant terminates wg0 in this same /24)
  • connector wg0 addresses, as /32
  • IoT peers on any tenant edge whose wg0 sits in the subnet

allocateConnectorIp() collects all three tenant-wide before choosing. Note the hub gateway is not guaranteed to be .1 — in the reference tenant below, edge-3 holds .1 and edge-2 holds .3.

IKEv2 pool (x.y.220.0/24) — the private DNS resolver lives at .254 of this range. That address is a /32 on lo on every edge, and each edge answers locally, so it is one address served by N independent CoreDNS instances that can disagree. Verifying a DNS change means querying every edge serving the zone, not one.

E2E peering (x.y.240.0/24) — this /24 holds two addresses per member:

  • the VPP tunnel IP, allocated .1 upward in join order
  • the bird BGP transport IP, at tunnel IP + 128

Bird runs on the Linux host, outside VPP, so it cannot use the VPP tunnel address directly. Both appear in the peer's allowed_ips. The transport address is carried as a /32 on lo inside bird's network namespace.

What is not in the tenant address space​

Three classes of address are legitimately outside the tenant /16, and anything that rewrites addresses must leave them alone.

ClassExampleNote
Branch LANs behind an edge192.168.15.0/24Customer-chosen. Carried in peer allowed_ips, never renumbered.
Networks behind a connectorconnectors.lan_subnetsCustomer-chosen. Advertised only by the hub whose leg actually carries them.
WAN / management addresses192.168.0.102, 10.0.16.56Underlay. Never overlay-routed.
VPP↔Linux link-locals169.254.x.x/30Per-tap point-to-point; identical across edges by design.
Host↔VPP data-plane entry172.31.254.x/30 on tap300Present in every VPP mode (VSNP-699).
A blind remap breaks working records to fix broken ones

The DNS drift reconciler (VSNP-1067) classifies every manual record into three buckets — inside the tenant's own /16 (correct), inside 100.64.0.0/10 but the wrong /16 (stale, remap), or outside the supernet entirely (external, never touched). The third bucket is not hypothetical: one live tenant holds manual records on 172.0.0.101 and 172.0.0.133. The other six cascade steps use a blind octet rewrite, which is safe only because they operate on values that are always tenant-internal.

Worked example: branch LAN to a connector-hosted service​

The reference path of VSNP-1017, traced live. Tenant Markus Demo, base 100.72.0.0/16.

 Raspberry Pi                Markus Home Edge          edge-2 (primary hub)      OHV-Connector-02
192.168.15.20 ──► loop0 192.168.15.1 ──► wg2 100.72.240.1 ──► wg0 100.72.200.2
(branch LAN) wg2 100.72.240.3 wg0 100.72.200.3 test-web :3010
BD 10: loop0 + G1 (bird transport .129)

Addresses, and which rule produced each​

AddressRule
192.168.15.0/24Branch LAN — customer-chosen, outside the tenant /16
100.72.240.1 / .2 / .3Peering /24, peering_octet 240, hosts in join order: edge-2, edge-3, MHE
100.72.240.129 / .130 / .131Bird transport IPs, tunnel IP + 128
100.72.200.3edge-2 wg0 gateway, connector /24, per-hub distinct host
100.72.200.1edge-3 wg0 gateway, same /24, different host
100.72.200.2Connector wg0 address, /32 from the same pool
100.72.220.254Private DNS resolver, ikev2_octet + .254

Hop 1 — branch LAN into VPP​

Bridge domain 10 on Markus Home Edge contains loop0 (the BVI, 192.168.15.1/24) and physical port G1. A host on G1 is bridged to the BVI and enters L3 there.

Hop 2 — MHE to edge-2 over the peering tunnel​

MHE's FIB resolves the service host across wg2:

100.72.200.2/32 fib:0 index:35 locks:2
path:[66] ip4 weight=1 pref=0 attached-nexthop: oper-flags:resolved,
100.72.240.1 wg2

oper-flags: resolved with an intact forwarding chain — no unresolved, no dpo-drop. The route is learned, not static:

100.72.200.2/32  unreachable [e2e_peer_1_t0 from 100.72.240.129] * (100) [i]
BGP.next_hop: 100.72.240.129

100.72.240.129 is edge-2's bird transport address — 240.1 + 128. The local_pref 100 marks this as the standby-preference path per VSNP-1051; edge-2 originates at 200.

Hop 3 — edge-2 to the connector over wg0​

edge-2 originates the prefix itself and hands it to BGP:

100.72.200.2/32  unreachable [e2e_advertised] * (200)
Type: static univ

Its WireGuard peers show cryptokey routing carrying both halves of the plan:

[0] wg0  allowed-ips: 100.72.200.2/32                                   ← the connector
[1] wg2 allowed-ips: 100.72.240.2/32 100.72.240.130/32 ← edge-3: tunnel + transport
[2] wg2 allowed-ips: 100.72.240.3/32 100.72.240.131/32
192.168.15.0/24 192.168.150.0/24 ← MHE: tunnel + transport + branch LANs

Peer [2] is the return path: the branch LAN prefixes sit on the leg toward MHE, so replies are encrypted to the right peer. Forward and return agree — no asymmetry.

Hop 4 — return path from the connector​

The connector holds two legs, and only the active one carries the branch:

primary  edge-2 139.99.217.230  active:true   allowed-ips 100.72.0.0/16, 192.168.15.0/24
standby edge-3 139.99.219.249 active:false allowed-ips 100.72.200.1/32, 100.72.200.2/32

This split is deliberate. If both hubs advertised the connector simultaneously, BGP could select edge-3 while the connector's traffic for that prefix is bound to the edge-2 tunnel — a black hole behind a healthy-looking control plane. On hub loss the orchestrator moves the prefix onto edge-3's leg instead.

Verification​

Branch-sourced ICMP from the BVI address, so the source is a genuine branch-subnet address:

$ vppctl ping 100.72.200.2 source loop0 repeat 4
116 bytes from 100.72.200.2: icmp_seq=1 ttl=63 time=29.5569 ms
...
Statistics: 4 sent, 4 received, 0% packet loss

And by name, from all three resolvers in the tenant:

dig @100.72.220.254 markustest.markus-demo.sl.internal
MHE → 100.72.200.2
edge-2 → 100.72.200.2
edge-3 → 100.72.200.2
Verifying a path

Two test methods do not work here and will produce a false failure:

  • curl from an edge's own Linux host. Its traffic leaves via the management gateway and never enters the overlay. ip route get 100.72.200.2 on Markus Home Edge returns via 192.168.0.1 dev eth0.
  • The portscan diagnostic. sourceInterface pins the egress path but not the source address — the probe enters VPP via tap300, so the source is always 172.31.254.x, which is in no peer's allowed_ips. Every port reads closed.

Use a real LAN host, or vppctl ping … source loop0.

Legacy space and the cascade​

Tenants created before unified addressing have no tenant_address_space row and fall back to fixed ranges:

ServiceLegacyUnified
App VPN10.10.0.0/24x.y.0.0/24
Connector10.200.0.0/24x.y.200.0/24
IKEv210.11.0.0/16x.y.220.0/24
Peering10.99.0.0/24x.y.240.0/24

cascadeAddressSpaceUpdate() renumbers a tenant onto unified space, rewriting seven tables — wireguard_app_server, wireguard_config, device_groups, connector_wireguard, edge_peering and its members, and (since VSNP-1067) tenant_dns_records — then marking the tenant's edges dirty.

Known limits and gaps​

Each of these was verified against live data. None is currently causing a failure; all are recorded so a future one is not a surprise.

1. The Device Group allocator does not know about the service reservations​

allocateSubnetForGroup() scans octet 3 from 1 to 253, excluding only octets already taken by another group. Its docblock lists the reserved values as "N=0, N=254, N=255" — it has no knowledge of the 200 / 220 / 221–239 / 240 / 241 reservations documented above. The 200th Device Group in a tenant would be allocated the connector backbone /24.

Headroom: the largest tenant today has 3 Device Groups, highest octet 3. Remote, but unenforced.

2. A cascade re-run would collapse every hub wg0 onto .1​

Cascade step 3 writes x.y.{connector_octet}.1/24 to every edge serial in the tenant, unconditionally. This predates VSNP-1016 connector HA, which requires each hub to hold a distinct host in that /24 and allocates them via getOrAllocateConnectorGatewayForEdge().

Live: edge-2 holds 100.72.200.3/24 and edge-3 holds 100.72.200.1/24. Re-running the cascade on this tenant today would set both to 100.72.200.1/24 — duplicate gateways, and edge-2's current address stomped.

This is the most likely of the six to bite, because the cascade is an operation someone may reasonably re-run.

3. Peering capacity is 127 members, and nothing enforces it​

The peering /24 is shared between tunnel IPs (allocated .1 upward) and bird transport IPs (tunnel + 128). allocateNextIP() scans .1–.254 with no knowledge of the transport half. So:

  • member 128 gets tunnel .128 → transport .256, not a valid address
  • member 129 gets tunnel .129, which is already member 1's transport address

Headroom: the largest peering today has 4 members.

4. MTGE tenant slots can run past the IKEv2 reservation into peering​

The per-slot IKEv2 pool is ikev2_octet + 1 + slot, and slot comes from mtge_devices.next_tenant_slot, which increments without a cap. Slot 19 yields octet 240 — the peering subnet.

Headroom: the single MTGE device in service is at next_tenant_slot 5. Fourteen more tenant assignments on that device reach the collision. This is the nearest of the design limits.

5. Seven edges still carry a legacy wg0 address inside an allocated tenant​

Sixteen wireguard_config rows still read 10.200.0.x/24. Nine belong to devices with no tenant assignment, where the legacy fallback is correct. Seven belong to tenants that hold a unified /16 — including Markus-Home-Edge, whose wg0 is 10.200.0.1/24 while its tenant holds 100.72.0.0/16. Confirmed on the data plane, not just in the database.

Harmless today: wg0 only matters where a connector or IoT peer terminates on that edge, and these have none. But it is the same class of drift VSNP-1067 built a report for, and there is no equivalent report for wg0.

Not a gap: the stale ikev2_virtual_ip_pool column​

Twenty wireguard_app_server rows read 10.11.0.0/16. This is the schema default on rows that have never used it — all twenty are tunnel_protocol = wireguard. The column is a stored override that does win over the derived value at batch build time, but the protocol-switch path refreshes it from getIkev2PoolCidr() when a row becomes IKEv2, and nulls it on the way back. All four live IKEv2 rows hold correctly derived pools. Recorded because the value looks alarming and is not.

Rules of thumb​

  • Derive, never hardcode. Use getServiceSubnet(), getVpnGateway(), getConnectorGateway(), getIkev2Pool(). A literal 100.64. in new code is a bug waiting for the second tenant.
  • Classify before you rewrite. Any code that remaps addresses must distinguish tenant-internal from external. Outside 100.64.0.0/10 means leave it alone.
  • A prefix lives on exactly one peer. Advertising a prefix from a hub whose leg does not carry it produces a black hole with a perfect-looking control plane.
  • Check every edge. One address, N resolvers; one tenant, N hubs. A green result from a single edge is not evidence.
  • A flag is a moment; a sequence number is a fact. A sync is asynchronous — the API returns immediately, the agent applies roughly a minute later. Read the agent log and the sequence number, not the instantaneous value.