Anvil! Networking

Alteeve Wiki :: How To :: Anvil! Networking

The Anvil! Cluster implements four main network types, each of which there could be one or more of.

Network Name	Prefix	Subnet	Used By	Description
Back-Channel Network	BCN	10.20{x}/16	All Machines	The BCN is used for cluster communications and monitoring of foundation pack equipment.
Internet-Facing Network	IFN	<user defined>	All Machines	The IFN is the connection between the servers hosted on the Anvil! and the user's main network. The subnet of this network is, therefor, defined by you.
Storage Network	SN	10.10{x}/16	Nodes, some DR	The SN is used for storage replication and nothing else. This is often connected back to back between subnodes in an Anvil! node. When connecting to a DR host
Migration Network	MN	10.199/16	Nodes	The MN is an optional back-to-back network link used between subnodes. When this network is available, when a hosted server live-migrates, the contents of the server's RAM is copied over this dedicated network. When this network is not available, RAM copy happens over the BCN (which, in some high load cases, could cause contention).

Back-Channel Network

Back-Channel Networks are the primary networks used by the various Anvil! cluster machines. Examples;

• Commands from the Striker dashboard are transmitted to the nodes and DR hosts via the BCN.
• Scancore used the BCN to communicate with UPSes, PDUs, IPMI out-of-band management interfaces, etc.
• Servers being live-migrated between subnodes, when the MN is not available, copies over the BCN

Most clusters have a single BCN, which uses the subnet 10.201/16. If there's need for a second BCN, it would be 10.202/16, and so forth, with the third digit of the second octet matching the network's sequence number (ie: BCN1 uses 10.201.x.y).

Device IP Addresses

Note: The IPs assigned to devices on the BCN match how IPs are assigned on other networks. The way IPs are assigned will be discussed in detail here.

In the BCN, SN and MN, the third octet of the IP indicates the device type, and the fourth octet defines the sequence number. The third octet will be between 1~9 for Strikers and foundation pack devices. The third octet of 10 and higher is assigned to nodes and DR hosts.

Lets lay this out in a table so that it makes more sense. For this example, 10.201/16 will be used as we're in the BCN section. Note that, where applicable, the same third and fourth octets will be used for the SN and MN also.

Device	Third Octet	Example (first device)
Ethernet Switches	1	10.201.1.1
Switched PDU	2	10.201.2.1
Managed UPS	3	10.201.3.1
Striker Dashboard	4	10.201.4.1
Striker IPMI	5	10.201.5.1

The octets 6 through 9 are not yet assigned to any device type.

Anvil! nodes start with the octet 10 for node 1, 12 for node 2, 14 for node 3, and so on. The reason for jumping by two is to allow for the next octet up to be used by the subnode's IPMI interface. Lets look at an example;

Node	Subnode	BCN 1 - IP	IPMI - IP
Node 1 (an-anvil-01)	an-a01n01	10.201.10.1	10.201.11.1
	an-a01n02	10.201.10.2	10.201.11.2
Node 2 (an-anvil-02)	an-a02n01	10.201.12.1	10.201.13.1
	an-a02n02	10.201.12.2	10.201.13.2
Node 3 (an-anvil-03)	an-a03n01	10.201.14.1	10.201.15.1
	an-a03n02	10.201.14.2	10.201.15.2
Node 4 (an-anvil-04)	an-a04n01	10.201.16.1	10.201.17.1
	an-a03n02	10.201.16.2	10.201.17.2
Node 5 (an-anvil-05)	an-a05n01	10.201.18.1	10.201.19.1
	an-a05n02	10.201.18.2	10.201.19.2

Note: Generally, DR Hosts are assigned to a node, matching its hardware. This isn't required, a DR host can protect servers from multiple nodes, but the IP assignment is based around peering DR hosts to nodes. As such, DR hosts have host names and IPs that link them to a node. If you plan to use a DR host for multiple nodes, you can connect it to which ever node you like. The naming and IPs are not used to enforce any actual linkages.

With the above note said, we'll show an example where there is a DR host per node.

Node	DR Host	BCN 1 - IP	IPMI - IP
an-anvil-01	an-a01dr01	10.201.10.3	10.201.11.3
an-anvil-02	an-a02dr01	10.201.12.3	10.201.13.3
an-anvil-03	an-a03dr01	10.201.14.3	10.201.15.3
an-anvil-04	an-a04dr01	10.201.16.3	10.201.17.3
an-anvil-05	an-a05dr01	10.201.18.3	10.201.19.3

 

Any questions, feedback, advice, complaints or meanderings are welcome.
Us: Alteeve's Niche!	Support: Mailing List	IRC: #clusterlabs on Libera Chat
© Alteeve's Niche! Inc. 1997-2023		Anvil! "Intelligent Availability™" Platform
legal stuff: All info is provided "As-Is". Do not use anything here unless you are willing and able to take responsibility for your own actions.