networking

Overview

Networks

IP Net	Description
10.1.0.0/24	IPMI network
10.1.3.0/24	OpenFlow (Control) network
10.1.1.0/24	(Cloud) Control network
134.60.64.0/24	Public network

Splitting up Public Network

In order to be more flexible with moving around floating IPs, the public network is splitted in four parts:

IP Net	Description	From	To	Gateway	Broadcast
134.60.64.0/26	Public Servers	134.60.64.2	134.60.64.62	134.60.64.1	134.60.64.63
134.60.64.64/26	ExtNet1	134.60.64.66	134.60.64.126	134.60.64.65	134.60.64.127
134.60.64.128/26	ExtNet2	134.60.64.130	134.60.64.190	134.60.64.129	134.60.64.191
134.60.64.192/26	ExtNet3	134.60.64.194	134.60.64.254	134.60.64.193	134.60.64.255

The openstack network node must be in each network, if used as external network for floating IPs, using the Gateway IP address.

Cloud Control Net Gateway

Setup one node as gateway (e.g. Network Node eth1 internal, br-ex external interface)

iptables --insert FORWARD -i eth1 -j ACCEPT
iptables --insert FORWARD -o eth1 -j ACCEPT
iptables --insert INPUT 1 -s 10.1.1.0/24 -i eth1 -j ACCEPT
iptables --table nat --append POSTROUTING --out-interface br-ex -j MASQUERADE
echo 1 > /proc/sys/net/ipv4/ip_forward

To make the ip forwarding persistent a) edit /etc/sysctl.conf and set net.ipv4.ip_forward = 1 b) save your iptables service iptables save.

Tell all other nodes to use this node as gateway

# route -n # lists current routes, check for consistency
# route add default gw 10.1.1.11 eth0

To make the default route persistent, edit the file /etc/sysconfig/network:

NETWORKING=yes
HOSTNAME=computenode14
GATEWAY=10.1.1.11

Hosts

IP Suffix	Node Name	Attached Networks
.10	Control Node	IPMI, Control, Public
.11	Network Node	IPMI, Control, Public
.12	Storage Node	IPMI, Control, Data
.13	SDN-Control Node	IPMI, Control, OpenFlow
.14	SDN Switch	IPMI, OpenFlow
.16	Cluster Management	IPMI, Control, Public
.20	XeonPhi	Public
.101	Computenode01	IPMI, Control
.1xx	...	IPMI, Control
.116	Computenode16	IPMI, Control
.117	Computenode17	IPMI, Control not yet configured
.151	Computenode01	IPMI entry point

VLANs

ID	Status	Fwd/Up /Cfg	Name	Type	Protocol	Ext. IP
10	Up	17/ 17/ 28	Topology Detec..	Port	-	- - - -
101	Up	17/ 17/ 28	Multicast/Broa..	Port	-	- - - -
102	Up	17/ 17/ 28	Multicast/Broa..	Port	-	- - - -
1000	Up	1/ 1/ 1	OpenFlow 1	Port	-	- - - 4
1100	Up	20/ 20/ 22	Control Networ..	Port	-	- - - -
1110	Up		Nuage Control	Port	-	- - - -
1120	Up		Nuage Data	Port	-	- - - -
1150	Up		omistack2 data network	Port	-	- - - -

|1200| Up | | OpenStack Tenants| Tagged | | | |... | Up | | OpenStack Tenants| Tagged | | | |1710| Up | | OpenStack Tenants| Tagged | | | |4094| Up | 17/ 17/ 28 | Unicast openfl.. | Port | - | - - - - |

VLAN ID 1200 - 1710 used for OpenStack Neutron.

Initial Setup

e-mail from Olivier/NEC

1 - IPMI
The IPMI addresses have been set on all nodes.
Network: 10.1.0.0/24
Default gateway: 10.1.0.254 (please, try to use this address,
else all IPMI interfaces will have to be modified)
   Control node   : 10.1.0.10
   Netw. node     : 10.1.0.11
   Storage node   : 10.1.0.12
   SDN controller : 10.1.0.13
   SDN switch     : 10.1.0.14
   Spare node     : 10.1.0.15
   compute node 1 : 10.1.0.101
   ..
   compute node 16: 10.1.0.116

All IPMI interfaces are reachable through web browser.
Login/password:
   Computes 1, 2, 3 and 4: Administrator/Administrator
   All other nodes       : ADMIN/ADMIN

2 - PFS5240 switch update
There has been no update of the firmware because it was already
at the latest available release: V6.0.0.0 (exactly: PF52L3PE-V6.0.0.0-13-20140612.img)

3 - Networks definition and addresses
Here are the networks definition and addresses:

OpenFLow network : 10.1.3.0/24 - VLAN 1000
   SDN contoller  : 10.1.3.13
   SDN switch     : 10.1.3.14
The OpenFlow network is used for OpenFlow communications between
the controllers and switches (currenlty, only 1 controller and 1 switch)

Control network  : 10.1.1.0/24 - VLAN 1100
   router : might be control node = 10.1.1.10
   Control node   : 10.1.1.10
   Netw. node     : 10.1.1.11
   Storage node   : 10.1.1.12
   SDN controller : 10.1.1.13
   compute node 1 : 10.1.1.101
   ..
   compute node 16: 10.1.1.116

Data network: managed by SDN

4 - PFS5240 switch configuration
The PFS switch has been configured according
to this network definition:

   - 2 ports for OpenFlow network - VLAN 1000 (first controller) and VLAN 2000 (second one)
     (only one port used now, a second port available in the future for a second controller)
     ports: 1 and 2

   - 22 ports for Control Network (legacy mode): VLAN 1100
     ports: 3 -> 24
     3 : Control node
     4 : Netw. node
     5 : Storage node
     6 : SDN controller
     7 : compute node 1
     ..
     22: compute node 16
     2 available ports

   - 27 ports (23 Gb, 4 10Gb) for OpenFlow
     ports: 25 -> 51
     25: compute node 1
     ..
     40: compute node 16
     8 available ports
     49: Netw. node (10 Gb)
     51: Storage node (10 Gb)
     2 available 10Gb ports

For the OpenFlow configuration, I have defined the VLANs which
are usually used by the NEC SDN controller. It might be different
with OpenDaylight.
Here are the VLANs configured:
OpenFlow network Topology detection: VLAN 10
Multicast/broadcast data transfer  : VLAN 101 first controller
                                      VLAN 102 second controller
Unicast data transfer              : VLAN 4094

5 - SDN controller
The OS has been installed: CentOS 6.5
hostname: sdn-1
system installed in RAID 1 (mdadm) on the 2 disks
grub installed on the 2 disks

/boot : /dev/md0 256 MB
remaining space (/dev/md1) used with LVM:
VG : vg_system
   lv_root : /    :  2 GB
   lv_usr  : /usr :  5 GB
   lv_home : /home: 10 GB
   lv_tmp  : /tmp :  2 GB
   lv_var  : /var :  5 GB
   lv_swap : swap : 24 GB

185 MB free in the VG.
I remind you that filesystems can be extended online.

root password: necINST.

eth0: OpenFlow network: 10.1.3.13
eth1: Control network : 10.1.1.13

basic server installation.
As we have not yet internet access from this server,
no update has been done.