NIC bonding (part II)

In the previous  article about NIC bonding the example referred to the "active-backup" bonding mode which ensures redundancy. Another very useful mode in NIC  bonding is "balance-alb", which is the most general mode for load balancing and throughput. Our purpose here is to create a new interface that will have twice the throughput of the two slave ports.

NIC bonding configuration

The first steps are the same and the two configurations differ only at a few points. Nevertheless for the sake of simplicity we shall describe all the steps. First, we assume that we know the IP address of the bonded interface (for this article’s purpose it will be 192.168.0.15) and the domain name of the server (here mydomain.com).  The process described in this article creates a bonded interface that works in the balance-alb mode (mode 6), which ensures load balancing and throughput. All the commands described here should be entered at the Linux console, so a basic understanding of the console and command line is necessary.  The commands will be coloured green and all the points that will need editing are marked red. Use the editor of your choice like nano or vim.

First update the system:

sudo apt-get update && sudo apt-get upgrade -y

Install the ifenslave package which ensures that two or more NICs can be bonded together.

sudo apt-get install ifenslave-2.6 -y

Edit your /etc/modules configuration file. This file describe which kernel modules are initialized at start-up. Ensure that the bonding module is loaded:

sudo vim /etc/modules

After editing the file should look like this:

# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with “#” are ignored.

loop
lp
rtc
bonding

The configuration of the bonding module is described in a separate file. Edit /etc/modprobe.d/bonding.conf file

sudo vim /etc/modprobe.d/bonding.conf

The contents of the file should be:

alias bond0 bonding
options bonding mode=6 miimon=100

Now  load the bonding kernel module:

sudo modprobe bonding

Backup the existing interfaces file, in case you will need to undo bonding.

sudo cp -f /etc/network/interfaces /etc/network/interfaces.nobond

Edit your interfaces configuration:

sudo vim /etc/network/interfaces

For example, to combine eth0 and eth1 as slaves to the bonding interface bond0 using a simple balance-alb setup, with eth0 being the primary interface:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto bond0
iface bond0 inet manual
    address 192.168.01.15 #EDIT HERE THE ACTUAL IP ADDRESS
    netmask 255.255.255.0
    network 192.168.0.0 # EDIT THE ACTUAL NETWORK ADDRESS
    broadcast 192.168.0.255 #EDIT HERE THE BROADCAST ADDRESS
    gateway 192.168.0.250 #EDIT HERE YOUR GATEGAY ADDRESS
    # bond0 may uses balance-alb. The  balance-alb mode is the most general for load balancing and throughput
    bond-mode balance-alb
    bond-miimon 100
    bond-slaves eth0 eth1
dns-nameservers 192.168.0.100 #EDIT HERE THE ACTUAL ADDRESS OF THE DNS SERVER
    dns-search mydomain.com

#eth0 - the first network interface
auto eth0
iface eth0 inet manual
hwaddress ether aa:bb:cc:dd:ee:ff #edit the mac address of eth0 here
bond-master bond0

#eth1 - the second network interface
auto eth1
iface eth1 inet manual
hwaddress ether aa:bb:cc:dd:ee:ff #edit the mac address of eth1 here
bond-master bond0

copy the interfaces.bonding file to interfaces:

sudo cp -f /etc/network/interfaces /etc/network/interfaces.bonding

Restart the networking service or reboot the system.

References

[1] Wikipedia - Link Aggregation

[2] Ubuntu Bonding

[3] Understanding NIC bonding with Linux

[4] Linux Ethernet bonding driver HOWTO

Posted in Programming | Tagged , , , , | Comments Off on NIC bonding (part II)

NIC bonding on Ubuntu server 14.04

NIC bonding or port trunking or link aggregation is the process where two or more network ports or NICs are combined, in order to perform as a single port [1]. The process can be applied to servers as well as to switches. The steps described below are an adaptation of a very helpful article found in "ubuntu help" [2], and was the outcome of a lots of searching in the web and experimentation. The process applies to Ubuntu server 14.04. The working modes of NIC bonding ensure fault tolerance, load balance or increase in network speed since the two or more interfaces are working in parallel and double or triple the speed. This last feature is very useful on servers that need high throughput. More details about the bonding modes can be found in [3] and [4].

NIC bonding configuration

First, we assume that we know the IP address of the bonded interface (for this article's purpose it will be 192.168.0.15) and the domain name of the server (here mydomain.com).  The process described in this article creates a bonded interface that works in the active-backup mode (mode 1), which ensures fault tolerance. All the commands described here should be entered at the Linux console, so a basic understanding of the console and command line is necessary.  The commands will be coloured green and all the points that will need editing are marked red. Use the editor of your choice like nano or vim. My own preference is vim.

First update the system:

sudo apt-get update && sudo apt-get upgrade -y

Install the ifenslave package which ensures that two or more NICs can be bonded together.

sudo apt-get install ifenslave-2.6 -y

Edit your /etc/modules configuration file. This file describe which kernel modules are initialized at start-up. Ensure that the bonding module is loaded:

sudo vim /etc/modules

After editing the file should look like this:

# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.

loop
lp
rtc
bonding

The configuration of the bonding module is described in a separate file. Edit /etc/modprobe.d/bonding.conf file

sudo vim /etc/modprobe.d/bonding.conf

The contents of the file should be:

alias bond0 bonding
options bonding mode=1 miimon=100

Now  load the bonding kernel module:

sudo modprobe bonding

Backup the existing interfaces file, in case you will need to undo bonding.

sudo cp -f /etc/network/interfaces /etc/network/interfaces.nobond

NIC bonding is simple once you understand the limitations of each mode. if you're working in an environment where switches support 802.3ad and you have no special needs, use that mode. Conversely, if you have no switch support and just want to increase throughput and enable fail-over, use balance-alb. Finally, if you just need a data replication link between two servers, balance-rr is the way to go.

Edit your interfaces configuration:

sudo vim /etc/network/interfaces

For example, to combine eth0 and eth1 as slaves to the bonding interface bond0 using a simple active-backup setup, with eth0 being the primary interface:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

#eth0 - the first network interface
auto eth0
iface eth0 inet manual
bond-master bond0
bond-primary eth0

#eth1 - the second network interface
auto eth1
iface eth1 inet manual
bond-master bond0

# The primary network interface
auto bond0
iface bond0 inet static
    address 192.168.0.15 #EDIT HERE THE ACTUAL IP ADDRESS
    netmask 255.255.255.0
    network 192.168.0.0 # EDIT THE NETWORK ADDRESS
    broadcast 192.168.0.255 #EDIT HERE THE ACTUAL BROADCAST ADDRESS
    gateway 192.168.0.250 #EDIT HERE THE ACTUAL GATEWAY ADDRESS
    bond-mode active-backup #This NIC bonding mode provides fault tolerance
    bond-miimon 100
    bond-slaves none
    # dns-* options are implemented by the resolvconf package, if installed
    dns-nameservers 192.168.0.100 #EDIT HERE THE ACTUAL ADDRESS OF THE DNS SERVER
    dns-search mydomain.com

Backup the interfaces file to interfaces.bonding file :

sudo cp -f /etc/network/interfaces /etc/network/interfaces.bonding

Some times the bonded interface does not start, so it should be started manually. In order to automate this process, the commands should be given at start-up. rc.local is a good spot for this. Edit /etc/rc.local file

sudo vim /etc/rc.local

ifdown eth0
ifdown eth1
ifup eth0
ifup eth1
#edit here the actual address of bond0
ifconfig bond0 192.168.0.15 up

This last command might alter the /etc/resolv.conf file and result in loosing the nameserver information. This can be avoided by editing the /etc/resolvconf/resolv.conf.d/base file

sudo vim /etc/resolvconf/resolv.conf.d/base

and assuming that your name-server's address is 192.168.0.100 the contents should be:

nameserver 192.168.0.100
search mydomain.com

Now restart the network service:

sudo service networking restart

If this last step does not work and produces error messages you should reboot the system.

sudo reboot

Happy NIC bonding!

References

[1] Wikipedia - Link Aggregation

[2] Ubuntu Bonding

[3] Understanding NIC bonding with Linux

[4] Linux Ethernet bonding driver HOWTO

Posted in Programming | Tagged , , , , | Comments Off on NIC bonding on Ubuntu server 14.04

Introduction to UNIX BASH shell programming II.

This is the second post of the "Introduction to Unix BASH shell programming" series. Initially, in order to be easier to follow the discussion, some facts concerning the file system structure will be presented. Subsequently,  some directory and other basic commands  will be discussed, like cp, mv, rm.

Some Unix File System issues

The UNIX file system has records which refer to files and directories. These records are called inodes. According to Wikipedia :

In a Unix-style file system, an index node, informally referred to as an inode, is a data structure used to represent a filesystem object, which can be one of various things including a file or a directory. Each inode stores the attributes and disk block location(s) of the filesystem object's data. Filesystem object attributes may include manipulation metadata (e.g. change, access, modify time), as well as owner and permission data (e.g. group-id, user-id, permissions).

ln - create link. Creates a new inode that points to the same data (address in the disk). This link is called a hard link. ln [-s] <file name> <link name>. When the option -s is used a soft or symbolic link link is created. The symbolic link does not point directly to the data, but to the original inode (much like the link in MS Windows)

Directory BASH commands

mkdir - create a new directory. mkdir <directory name>. The directory name may be sdeclared as an absolute or relative path.

rmdir - remove directory.  Delete a directory. rmdir <directory name>. The directory name may be declared as an absolute or relative path, but in ant case it must be empty in order to be removed.

Basic commands

cp - copy file(s) or directory(ies).  cp [-i] [-r] [-f] <source file(s)> <target file>.  Any combination of files or directories may be used as source. The source files are separated by space. The target could be a file or a directory. In the case of multiple sources the target is a directory. The names of files and directories may be declared using the absolute or relative path. The copy action creates a new inode with the same contents. Thus, a change in either the source or destination file does not reflect to the the other, in contrast with the hard or symbolic links. The -f option does not ask for a permission to overwrite the destination file, when it already exists.

mv - move file(s) or directory(ies). mv [-i] [-r] [-f] <source file(s)> <target file>.This command behaves the same as the copy command except that it removes (deletes) the source object. If the source and destination file are located in the same directory, then the result of the command is to rename the source file giving it the name of the destination.

rm - remove file(s) or directory(ies).  rm [-i] [-r] [-f] <source file(s)> <target file>. The -r option removes recursively the contents of the target directory and all subdirectories and files. Be very careful with this command. For example, rm -rf /* might delete all the contents of your disk.

Posted in Programming | Tagged , , | Comments Off on Introduction to UNIX BASH shell programming II.

Introduction to UNIX BASH shell programming I.

UNIX history

In this series of posts I shall try to outline some of the most important items of BASH SHELL programming and provide examples and exercises. I have been teaching UNIX & BASH shell scripting in the Academia since  2000. Throughout these years I have expanded my experience and knowledge on UNIX & BASH shell scripting and thus I am able to elaborate numerous examples on several shell programming aspects.

Nowadays, it is a common knowledge that UNIX goes back to the late 60s and ATT where a team of bright engineers developed a multiuser , multitasking network capable secure Operating System. The initial version of UNIX was coded using the PDP-7 machine language but in the early 70s after the introduction of the C language it was recoded and ported to several other machines.  Linux is a UNIX compatible OS initially developed by Linus Torvalds. Since that time numerous new Linux distributions have emerged based on the original Linux kernel. Some of them are commercial while others are commuted to the idea of  open source free software.

The powerful feature of UNIX is the console and the shell. Of course one would argue that the GUI is valuable for everyday use. I could not agree more. The GUI is the tool for workstations but for the case of a server a GUI will just consume resources and nobody will ever see the fancy windows and colors. Additionally, the scripting capabilities of the shell give the ability to automate management and maintenance tasks.

 

The BASH SHELL

Simply put, the SHELL is a piece of software that realizes an interface between the user and the Operating System (the kernel). In a GUI the shell starts by starting the "konsole" or "terminal" application etc. Without a GUI the SHELL is the black screen that appears after login (the same stands for SSH). The user sees a '$' or '#' and a cursor waiting for the user to type a command.

The user may type a command or the name of a script and press [ENTER]. Then the command is interpreted by the shell and passed to the OS. The OS performs the required actions and responds (or not) with the command output or an error message.

The SHELL ecosystem includes several variations of the original "standard shell". To name a few there is "korn shell" ( ksh), Z shell, zsh, C shell (csh or tcsh) and Bourne shell (bash). The difference between them are mainly the programming features, the command syntax and start-files. The bash shell was written by Brian Fox for the GNU Project.

 

The first commands - Navigating

While describing the syntax and functionality of the shell commands the [  ] will be using ed to notify that the item included is optional (the [ ] are not used while typing the command ) and < > to notify that something is obligatory. The general command syntax is:

<command name>   [-options]   [arguments]

The options are usually letters or numbers that define  the behavior the specific command. They are optional to use and they are distinguished by the '-' sign. The options may be grouped in any possible way. For example the commands   ls -l -a, ls -a -l, ls -al and ls -la have exactly the same result and output. The arguments are the entities (files, directories, users, data etc) that the command uses as input or acts upon. They are optional too. The options and arguments  are separated by space.

pwd - print working directory. The command returns the current full path.

cd - change the shell working directory.  cd  [-L]  [-P] [dir]. Change the current directory to dir. The default DIR is the value of the user's HOME directory, which is stored in the HOME shell variable.
Options:
-L force symbolic links to be followed.
-P use the physical directory structure without following symbolic links.

ls - list directory contents. ls [OPTION]... [FILE]...  The ls command may have a lot of options and behave accordingly. For more info, the user my refer to the relevant man page or the unix.com page
Options:
-a, --all do not ignore entries starting with .
-l, use a long listing formatCheck
-s, --size with -l, print size of each file, in blocks
-i, --inode with -l, print the index number of each file
-d, --directory list directory entries instead of contents, and do not dereference symbolic links
One important piece of information to keep in mind is the output format i.e. the names of the columns. This information may be quite useful when developing filters and scripts. For example in the commonly used form ls -al the first column describes the mode of the file, the second the number of the hard links for this file, the third and fourth the user and the group that own the file and then there may be the creation date or last touch date, the size and the name of the file.

Posted in Programming | Tagged , , | Comments Off on Introduction to UNIX BASH shell programming I.

The Tao of Programming

Something mysterious is formed, born in the silent void. Waiting alone
and unmoving, it is at once still and yet in constant motion. It is
the source of all programs. I do not know its name, so I will call it
the Tao of Programming. read more...

Posted in Programming | Comments Off on The Tao of Programming