SMB3 Multi-Channel with Samba on Red Hat Gluster Storage (Technology Preview)
Caveat - Technology Preview
Red Hat Gluster Storage 3.1.3 introduces SMB3 multi-channel as a Technology Preview feature: .
This feature is not yet supported in production environments, and may result in data corruption.
What is SMB3 multi-channel?
Multi-channel is an SMB3 protocol feature that allows the client to bind multiple transport connections into one authenticated
SMB session. This allows for increased fault tolerance and throughput. The client chooses transport connections as reported
by the server and also chooses over which of the bound transport connections to send traffic. This lets I/O operations for a given file
handle can span multiple network connections. An SMB multi-channel session is considered valid as long as at least one of its channels are up.
Clients supporting multi-channel include Windows 8 and newer and Windows Server 2012 and newer.
Configuring Samba for multi-channel
To enable multi-channel, edit the smb.conf file and set the value of the server multi channel support option to yes in the [global] section. Multi-channel is disabled (set to no) by default. As a prerequisite, make sure that the server max protocol setting in smb.conf is not set to anything lower than the default of SMB3: A value of e.g. SMB2 will effectively disable multi-channel.
Note: While a changed value for the server multi channel support parameter will immediately be effective for newly connected clients, one may consider restarting the samba service after such a change, depending on how the already connected clients should be treated.
Preparing the RHGS system for multi-channel
Because multi-channel sessions must always connect to a single node only, a Red Hat Gluster Storage system currently requires a special setup in order to safely support multi-channel: CTDB public addresses (aka VIPs) should be disabled in order to prevent IPs bound to a multi-channel session from being moved to a different node. This means that the failover features of clustered Samba in RHGS cannot be used at the same time as multi-channel.
Note: This limitation may be removed in future versions of Red Hat Gluster Storage.
To configure a non-VIPs setup, make sure there is no CTDB_PUBLIC_ADDRESSES setting in the /etc/sysconfig/ctdb file on any of the CTDB nodes. Furthermore, the cluster nodes should offer multiple client-facing network interfaces in addition to CTDB's node-internal interface. To complete the configuration, statically configure IP addresses on these additional client-facing NICs.
Example:
Assume there two nodes node0 and node1 in the cluster. Let eth0 be CTDB's node-internal interface configured with addresses 172.16.1.201 and 172.16.1.201. On both nodes, assume eth1 and eth2 are the additional client-facing NICs. Configure eth1 with 192.168.100.201/24 and eth2 with 192.168.200.201/24 on node0, and configure eth1 with 192.168.100.202/24 and eth2 with 192.168.200.202/24 on node1.
Variant:
Instead of manually setting the above IPs, one can still have CTDB set these as public addresses. This is achieved by creating the following public_addresses files on the two nodes:
node0:~$ cat /etc/ctdb/public_addresses
192.168.100.201/24 eth1
192.168.200.201/24 eth2
and
node1:~$ cat /etc/ctdb/public_addresses
192.168.100.202/24 eth1
192.168.200.202/24 eth2
And in this case activate the setting CTDB_PUBLIC_ADDRESSES=/etc/ctdb/public_addresses in /etc/sysconfig/ctdb instead of removing it. With this setting, CTDB will take care of assigning the corresponding addresses.
Using and Verifying Multi-Channel from Windows
After configuring RHGS for multi-channel, Windows 8 and newer should use multi-channel. Multi-channel connections can be listed with this powershell command on windows:
Get-SmbMultichannelConnection
Get-SmbMultichannelConnection -IncludeNotSelected
Note that when connecting to a share, Windows will detect and bind additional interfaces as available, but only switch to doing traffic over them as serious I/O is coming in over SMB. For example, just listing a directory will not usually trigger traffic over multiple interfaces, but copying a large file will.