Table of Contents
- 3.1. Mandriva Linux Installation Wizard
- 3.2. IGGI Server Server Post-Installation Configuration
-
- 3.2.1. Copying and Preparing Media
- 3.2.2. Install directory
- 3.2.3. Server configuration
- 3.2.4. Configuring DNS
- 3.2.5. Configuring NIS
- 3.2.6. Configuring DHCP
- 3.2.7. Configuring PXE
- 3.2.8. Setup administration
- 3.2.9. Compute
- 3.2.10. Configuring Postfix (disable by default)
- 3.2.11. OAR database
- 3.2.12. End of the Automated Configuration
- 3.2.13. drakcluster to register MAC address
- 3.3. Collecting MAC Addresses (not mandatory but usefull)
IGGI is based on Mandriva Linux 2006. So, the installation procedure is the same, except for some points explained below. Be careful to follow them to get a proper installation.
If your hardware is able to boot from a CD-ROM drive, insert your IGGI CD and boot the server.
Note that very simple steps are not described in this documentation. i.e: Keyboard, mouse configuration. In order to get a proper cluster server installation, please carefully follow the steps in the order they are presented.
Once your are in the graphical part of the installation, answer the first questions about your language and the license.
Follow the installation steps by answering default selections, then choose the "Standard" security level. If you choose High security level, you will encounter problem with shorewall firewall.
Now we must create partitions. The simplest way is to "Auto Allocate". First, click on "Clear all" to clean the old partitions from the hard disk, next click on the "Auto Allocate" button. Then, click on the "Done" button to validate.
If you want create the partitions by yourself, your "/" size must be quite big, about 2GB. You can create a partition of 1GB where you will copy the IGGI cdrom mounted as /var/install. You must isolate "/var" on its own partition, for safety reasons.
When you have finished, click on the "Done" button.
-
"swap", about 500MB.
-
"/", about 2GB.
-
"/var/install", 800MB.
-
"/var", about 1GB or more.
-
"/home", as big as possible.
Now, you can format all the partitions by clicking on the "OK" button.
Now you have to choose which packages to install. We are installing a IGGI server, so select these ones:
-
"Cluster Server"
-
"OAR Scheduler"
-
"Monitoring server"
-
"Administration"
-
"Duplication"
-
"Icewm Graphical Desktops" : Select it if you want a graphical environment.
You can adjust the packages you want to install using the individual package selection feature that allows you to add or remove packages.
This is the most crucial decision point for the security of your IGGI system: you have to enter the root password. Root is the system administrator and is the only user authorized to make updates, add users, change the overall system configuration, and so on. In short, root can do everything! That is why you must choose a password that is difficult to guess.
Clicking on the No password button will let you bypass root's password, but we strongly advise you against this. Since root can overcome all limitations and unintentionally erase all data on partitions by carelessly accessing the partitions themselves, it is important that it is difficult to become root.
The password should be a mixture of alphanumeric characters and at least 8 characters long. Never write down the root password --- it makes it far too easy to compromise a system. One caveat: do not make the password too long or complicated because you must be able to remember it!
The password will not be displayed on screen as you type it in, all symbols you type will be replaced by stars (*). To reduce the chance of a blind typing error you will need to enter the password twice. If you do happen to make the same typing error twice, this "incorrect" password will be the one you will have use the first time you connect.
You have to choose an IP address for your Ethernet card. Select a "end of range" IP address because nodes are (by default) at the "beginning of range". i.e. : 12.12.12.253 for server, so nodes addresses will be from 12.12.12.1 to 12.12.12.250.
If you have two network interfaces, you must use eth0 (the first you will configure) to communicate with your nodes (a.k.a. local network), and eth1 to reach your enterprise network.
Do not forget to add a domain name at the end of the hostname. i.e. : server.iggi.com.
The figure below shows a configuration using two network interfaces, so the default gateway is set to your internal gateway.
Most steps will write a log of the operations performed in the /tmp/log/ directory. The log file will be named according to the step being performed, for example: setup_install_cluster.pl.log for the "Copy Installation Source" step.
Once your server has finished his installation, after the first boot a set of scripts will reconfigure it from scratch. Do not worry about errors seen during the first boot of your server, the server is not yet fully configured! All settings used by the scripts are stored in /etc/clusterserver.conf . Edit it if you do not like the default configuration. Log as root, open a terminal, and type the following:
[root@iggi root]# setup_auto_server
it's popup a dialog box called draktab_config to be able to edit /etc/clusterserver.conf via a graphical method. Adjust VAR to fit your need, anc click the apply button. Once done, you can close this dialog box.
Second step it to show you what will be done by the setup_auto_cluster script.
Sometimes, the server cannot read the CD-ROM. If that happens, break the script using CTRL+C and reload the CD-ROM using the following command:
[root@iggi root]# eject; eject -t; mount /mnt/cdrom; setup_auto_server
The configuration will now be fully automatic. The first step is to copy the whole CD-ROM to the server in /var/install/cluster directory, so it takes several minutes.
Copying and preparing media (setup_install_cluster.pl)
|---------------------------------| | AUTO SETUP CLUSTER SERVER ! | |---------------------------------| You can adjust your var in /etc/clusterserver.conf This script is going to automaticaly execute in order: setup_install_cluster.pl doall setup_server_cluster.pl gennodeone setup_server_cluster.pl doall setup_dns.pl master setup_nis.pl set_server setup_dhcpdconf_server.pl doall setup_pxe_server.pl build setup_pxe_server.pl boot autohttp setup_admin.pl doall setup_compute.pl doall setup_postfix.pl set oar_db_init setup_add_nodes_to_dhcp.pl -n number_of_nodes Be sure that the CDROM is present (first launch) (press ENTER when ready)
This will create an ftp account, login: install without password, and create an install directory /var/install/cluster, and finally allow access to IGGI installation directory through NFS and HTTP. Server will setup his urpmi database on this directory.
Install configuration
--------------------------------- LAUNCH: setup_install_cluster.pl LOG can be found in: /tmp/log/setup_install_cluster.pl.log --------------------------------- - Add user FTP install, with no password - Copying CD to /var/install/cluster `/mnt/nfs/install' -> `/var/install/cluster/install' `/mnt/nfs/install/stage2' -> `/var/install/cluster/install/stage2' `/mnt/nfs/install/stage2/VERSION' -> `/var/install/cluster/install/stage2/VERSION' `/mnt/nfs/install/stage2/mdkinst.kernels' -> `/var/install/cluster/install/stage2/mdkinst.kernels' `/mnt/nfs/install/stage2/mdkinst.clp' -> `/var/install/cluster/install/stage2/mdkinst.clp' `/mnt/nfs/install/stage2/rescue.clp' -> `/var/install/cluster/install/stage2/rescue.clp' ................ - Removing old installation media removing medium "http 1 (http1)" found 0 headers in cache removing 0 obsolete headers in cache wrote config file [/etc/urpmi/urpmi.cfg] - Adding media /var/install/cluster copying hdlists file... ...copying done added medium (cluster_m1) computing md5sum of existing source hdlist (or synthesis) copying source hdlist (or synthesis) of " (cluster_m1)"... ...copying done examining hdlist file [/var/cache/urpmi/partial/hdlist. (cluster_m1).cz] examining pubkey file of " (cluster_m1)"... performing second pass to compute dependencies examining hdlist file [/var/lib/urpmi/hdlist. (cluster_m1).cz] built hdlist synthesis file for medium " (cluster_m1)" found 0 headers in cache removing 0 obsolete headers in cache wrote config file [/etc/urpmi/urpmi.cfg] - Adjusting /etc/exports - service nfslock reload Stopping NFS locking: [ OK ] Stopping NFS statd: [ OK ] Starting NFS statd: [ OK ] - Creating link for acces rpm through HTTP setup_install_cluster.pl finish with SUCCESS (press ENTER key to continue)
This script will setup the basic network configuration (IP, PAM, Node list, keys....).
Server configuration (setup_server_cluster.pl)
--------------------------------- LAUNCH: setup_server_cluster.pl LOG can be found in: /tmp/log/setup_server_cluster.pl.log --------------------------------- Setting up a Cluster server with this configuration |----------------------------------------------------------- | Hostname | iggi.guibland.com |----------------------------------------------------------- | Domainname | guibland.com |----------------------------------------------------------- | Nis domain | guibland.com |----------------------------------------------------------- | Nis Server | iggi.guibland.com |----------------------------------------------------------- | NFS server | iggi.guibland.com |----------------------------------------------------------- | NIS home directory | /home/nis |----------------------------------------------------------- | User SSH dir | /root/.ssh |----------------------------------------------------------- | Nodesfile | /etc/nodes_list |----------------------------------------------------------- | UrpmiGroup | cluster |----------------------------------------------------------- | Range For node | 1 - 128 |----------------------------------------------------------- - Configure Smartmontools - service smartd restart Shutting down smartd: [ OK ] Starting smartd: [ OK ] - kill anoying gam_server - service mysqld-max restart Stopping : [ OK ] Starting MySQL: [ OK ] - Checking ssh dir for user - Generating New ssh key Generating public/private dsa key pair. Your identification has been saved in /tmp/server/id_dsa. Your public key has been saved in /tmp/server/id_dsa.pub. The key fingerprint is: 7f:6a:a9:7f:8e:30:b5:50:98:8d:62:ce:69:2c:48:72 [email protected] - Generating New auth key - Running nisdomainname guibland.com - Generating nisdomain file for tftp - Creating /home/backup - Copying file in /tmp/server - Copying file in normal dir - Copy /var/lib/oar/.ssh/id_dsa.pub in /var/lib/tftpboot directory - Updating /var/lib/tftpboot files - no /var/install/cluster/doc/cluster found :-( - Cleaning Working Directory - Backup of /etc/nodes_list configuration - Generating nodes list setup_server_cluster.pl finish with LOG Look in /tmp/log/setup_server_cluster.pl.log for more information (press ENTER key to continue)
This script will configure a DNS master server that will translate node's names to IP addresses and vice-versa. The DNS server could be also a forwarder on your master DNS for resolving Intranet, Extranet, Internet names and IPs. We now used a chrooted Bind server.
Configuring the DNS service (setup_dns.pl)
--------------------------------- LAUNCH: setup_dns.pl LOG can be found in: /tmp/log/setup_dns.pl.log --------------------------------- Setting up DNS server with this configuration |----------------------------------------------------------- | Short Hostname | iggi |----------------------------------------------------------- | IP of DNS server: | 12.12.12.253 |----------------------------------------------------------- | IP External: | 12.12.12.253 |----------------------------------------------------------- | Add search: | |----------------------------------------------------------- | Domainname: | guibland.com |----------------------------------------------------------- | Admin nodename: | node |----------------------------------------------------------- | First node: | node1 |----------------------------------------------------------- | Last node: | node128 |----------------------------------------------------------- | IP range in DNS: | 12.12.12.1 - 12.12.12.128 |----------------------------------------------------------- | File of DNS server: | /var/lib/named/var/named/zone |----------------------------------------------------------- | Work dir: | /tmp/dnssetup |----------------------------------------------------------- - Checking IP of SERVER in RANGE IP - Creating /tmp/dnssetup/db.localhost - Creating /tmp/dnssetup/db.127.0.0.1 - Creating /tmp/dnssetup/trusted_networks_acl.conf - Creating /tmp/dnssetup/bogon_acl.conf - Creating /tmp/dnssetup/named.conf - Setting /tmp/dnssetup/hosts - Setting /tmp/dnssetup/hosts - Will be a Master DNS :-) - Creating /tmp/dnssetup/db.12.12.12.hosts - Creating /tmp/dnssetup/db.guibland.com.hosts - Creating /tmp/dnssetup/rndc.conf - Setting /tmp/dnssetup/resolv.conf - Backup of /etc/named.conf configuration - Copying DNS server files in correct place - service named start Starting named: [ OK ] setup_dns.pl finish with SUCCESS (press ENTER key to continue)
This script will configure the NIS (Yellow Pages) service, which is an authentication service. This allows users to log in on the nodes using the same environment. It's also configure autofs to be able to automount user home on all nodes throug NFS.
Configuring the NIS service (setup_nis.pl)
--------------------------------- LAUNCH: setup_nis.pl LOG can be found in: /tmp/log/setup_nis.pl.log --------------------------------- Setting up NIS server with this configuration |----------------------------------------------------------- | NIS server: | iggi.guibland.com |----------------------------------------------------------- | Domainname: | guibland.com |----------------------------------------------------------- | Nis Domain: | guibland.com |----------------------------------------------------------- | NIS home directory: | /home/nis |----------------------------------------------------------- | NIS DIR makefile: | /var/yp |----------------------------------------------------------- | NFS server: | iggi.guibland.com |----------------------------------------------------------- - Testing NISdomain value - Backup of /etc/auto.master configuration - Adjusting /etc/auto.master - Permission for all user to go in /etc/auto.home - Running nisdomainname guibland.com - Updating /var/yp to add autofs - You can add a user or remove it with: deluserNis and adduserNis command - do make /var/yp/Makefile make: Entering directory `/var/yp' gmake[1]: Entering directory `/var/yp/guibland.com' Updating passwd.byname... Updating passwd.byuid... Updating group.byname... Updating group.bygid... Updating hosts.byname... Updating hosts.byaddr... Updating rpc.byname... Updating rpc.bynumber... Updating services.byname... Updating services.byservicename... Updating netid.byname... Updating protocols.bynumber... Updating protocols.byname... Updating auto.master... Updating auto.home... # mail netgrp publickey networks ethers bootparams printcap \ # amd.home auto.master auto.home auto.local passwd.adjunct \ # timezone locale netmasks gmake[1]: Leaving directory `/var/yp/guibland.com' make: Leaving directory `/var/yp' - Backup of /etc/yp.conf configuration - Updating /etc/yp.conf - Backup of /etc/sysconfig/network configuration - Set nisdomain to: guibland.com - Set automount to nis in /etc/nisswitch.conf - Creating /home/nis directory - Adjusting /etc/exports - service nfslock restart Stopping NFS locking: [ OK ] Stopping NFS statd: [ OK ] Starting NFS statd: [ OK ] - service ypserv restart Stopping YP server services: [ OK ] Starting YP server services: [ OK ] - service ypbind restart Shutting down NIS services: [FAILED] Binding to the NIS domain... [ OK ] Listening for an NIS domain server: iggi.guibland.com - service nfs reload (press ENTER key to continue)
This script configures the DHCP service used for giving addresses, gateway, tftpserver, domain name to your nodes. The DHCP configuration also performs the basic PXE configuration.
Configuring the DHCP service (setup_dhcpdconf_server.pl)
---------------------------------
LAUNCH: setup_dhcpdconf_server.pl
LOG can be found in:
/tmp/log/setup_dhcpdconf_server.pl.log
---------------------------------
Setting up dhcpd.conf with this configuration:
|---------------------------------------------------------
| admin domain name | guibland.com
|---------------------------------------------------------
| admin server ip | 12.12.12.253
|---------------------------------------------------------
| DHCP range ip | 12.12.12.1 - 12.12.12.128
|---------------------------------------------------------
| TFTPSERVER | 12.12.12.253
|---------------------------------------------------------
- Setting dhcpd.conf
- tftpserver
- admin bloc :
- subnet
- option routers
- option domain-name
- option domain-name-servers
- range
- service dhcpd stop
Shutting down dhcpd: [ OK ]
- service dhcpd start
Starting dhcpd: [ OK ]
(press ENTER key to continue)
This script prepares the default PXE configuration files. It takes vmlinuz and all.rdz from /var/install/cluster/isolinux/altX directory, and prepare the PXE default configuration file stored in /var/lib/tftpboot/X86PC/linux/pxelinux.cfg/ directory.
Configuring the PXE service (setup_pxe_server.pl)
---------------------------------
LAUNCH: setup_pxe_server.pl
LOG can be found in:
/tmp/log/setup_pxe_server.pl.log
---------------------------------
- Preparing common configuration file
- Preparing Default PXE Menu
- Preparing PXE Message
- Creating pxe images directories
- Preparing auto installation file /var/install/cluster/install/stage2/auto_inst.cfg.pl
- Backup of /var/install/cluster/install/stage2/auto_inst.cfg.pl configuration
- Preparing Network installation Entry
- Preparing KA PXE entry
- Preparing Dolly PXE entry
- Preparing Network installation Entry 2.6.12-26mdksmp
- Preparing KA PXE entry 2.6.12-26mdksmp
- Preparing Dolly PXE entry 2.6.12-26mdksmp
- Preparing rescue mode
- Preparing PXE for cards
- Backup of /etc/pxe.conf configuration
- Setting /etc/pxe.conf file
/etc/pxe.conf
- Interface
- address
- MTFTP address
- Domain
- Domain
- Restart PXE service
Stopping PXE server [ OK ]
Starting PXE server [ OK ]
- Garbage collector in action
- Entering setup mode
- Patching /var/lib/tftpboot/X86PC/linux/pxelinux.cfg/default
PXE Server is now set to boot on 'autohttp' Entry
setup_pxe_server finish with SUCCESS
(press ENTER key to continue)
This script prepares all needed to administrate the IGGI cluster: gexec, ka-tools, gsh, wulfstat, tentakel, taktuk2, urpmi parrallel... All needed var are set in /etc/profiles.d/cluster.sh.
Configuring all administration tools (setup_admin.pl)
---------------------------------
LAUNCH: setup_admin.pl
LOG can be found in:
/tmp/log/setup_admin.pl.log
---------------------------------
Setting up Cluster Admin server with this configuration
|-----------------------------------------------------------
| Hostname | iggi.guibland.com
|-----------------------------------------------------------
| IP of DNS server | 12.12.12.253
|-----------------------------------------------------------
| Domain Admin: | guibland.com
|-----------------------------------------------------------
| urpmi parallel: | /etc/urpmi/parallel.cfg
|-----------------------------------------------------------
| Urpmi Group: | cluster
|-----------------------------------------------------------
- Creating Clusterit list node file
- Setting basic conf for Clusterit
- Creating Gsh configuration file
- Reseting basic conf for GEXEC
- Configuring tentakel
- Configuring pssh
ENV ok
- Reseting rshp and mput
- Configure wulfstat
- Configuring dssh
- add node1.guibland.com from cluster urpmi group
node1.guibland.com is ACTIVE
setup_admin.pl finish with SUCCESS
(press ENTER key to continue)
Configure all compute services: MPICH, LAM, Mpich2, distcc, and copy all needed files in /var/lib/tftpboot
--------------------------------- LAUNCH: setup_compute.pl LOG can be found in: /tmp/log/setup_compute.pl.log --------------------------------- Setting up Cluster server with this configuration |----------------------------------------------------------- | Hostname | iggi.guibland.com |----------------------------------------------------------- | IP of DNS server | 12.12.12.253 |----------------------------------------------------------- | Domain | guibland.com |----------------------------------------------------------- | Nodesfile | /etc/nodes_list |----------------------------------------------------------- | Lam | /etc/lam/lam-bhost.def |----------------------------------------------------------- | Mpi | /usr/share/mpich/machines.LINUX |----------------------------------------------------------- - Backup of /etc/nodes_list configuration - Backup of /etc/profile.d/cluster.sh configuration - Setting Distcc - Configure Mpi - Backup of /usr/share/mpich/machines.LINUX configuration - Configuring Lam - Backup of /etc/lam/lam-bhost.def configuration - Configure Mpd.conf - Configure Mpich2 - Copy current config in /var/lib/tftpboot directory - Running nisdomainname guibland.com - Generating nisdomain file for tftp setup_compute.pl finish with SUCCESS (press ENTER key to continue)
The /usr/bin/setup_postfix.pl script setups a Postfix mail server based on your domain name. Users can use mutt to send or receive mails.
--------------------------------- LAUNCH: setup_postfix.pl LOG can be found in: /tmp/log/setup_postfix.pl.log --------------------------------- Setting up POSTFIX server with this configuration |----------------------------------------------------------- | Postfix server | iggi.guibland.com |----------------------------------------------------------- | Postfix Domain | guibland.com |----------------------------------------------------------- | Postfix myorigin | guibland.com |----------------------------------------------------------- - Setting configuration /etc/postfix/main.cf file for Postfix server - Backup of /etc/postfix/main.cf configuration - service postfix restart Shutting down postfix: [FAILED] Starting postfix: [ OK ] setup_postfix.pl finish with SUCCESS (press ENTER key to continue)
OAR is our batch scheduler, it works with a MySQL database. By default the Admin login for the MySQL is root. There is no default password, so you can add one. Morever to be able to access to MySQL database from node (to update node OAR status) MySQL is started without the options "--skip-networking".
oar_db_init script
|-------------------------------------------|
| Initialisation of oar Database |
| Admin login is root user without password |
|-------------------------------------------|
Using /usr/lib/perl5/vendor_perl/5.8.7//oar_jobs.sql for the database creation
## Initializing OAR MySQL database ##
Retrieving OAR base configuration for OAR configuration file:
MySQL server hostname: localhost
OAR base name: oar
OAR base login: oar
Please enter MySQL admin login information:
Admin login:
Admin password:
Enter password:
This is the end of the automated installation of the server.
End of auto-configuration
- Restarting gmond, gmetad, on server Starting GANGLIA gmond: [ OK ] Shutting down GANGLIA gmetad: [ OK ] Starting GANGLIA gmetad: [ OK ] Server READY, now install a client Install a golden node and replicate it with ka tools (press ENTER key to exit)
Now your server is up and running, you can install your first node to prepare the ka-based replication. This step allows you to deploy a node to others using parallel copying technology. You can also use Section 12.5.2, “Get stage2 via Dolly”, or the Chapter 11, Diskless node mode to install your nodes.
Now you can launch drakcluster.pl. For more information about drakcluster, see Chapter 9, Drakcluster User guide. You can quickly add the MAC address of you first node into the /etc/dhcpd.conf configuration file, just do a 'right clic' on the column 'Known in DHCP' and choose the "Add nodes in dhcpd.conf' menu entry.
A popup dialog box will appear. If you want to add just one MAC address in the dhcp configuration, clic the 'OK' button.
Then another popup dialog box will appear. The server wait for one node. Boot this node through the network (PXE), and the server will automatically discover the MAC address, add it into the DHCP configuration, and restart the dhcpd server.
Now the MAC address of the node is registered, drakcluster will be updated to reflect this situation (it can take some times).
This script helps you in collecting the MAC addresses and assign them a static IP address and a hostname. When the script is running, you just have to power-on your remaining nodes. When a node is making a DHCP request, the script plays a small sound on your buzzer. When all the nodes are detected, the script exits and saves the DHCP configuration.
When all nodes have been discovered, you can shut them all down.
Collecting MAC addresses (setup_add_nodes_to_dhcp.pl)
[root@guibpiv ~]# setup_add_nodes_to_dhcp.pl -n 4 This script works on a backup version of /etc/dhcpd.conf It can be found in /etc/dhcpd.conf.add_nodes The server is now collecting the MAC addresses. Please bootup your nodes, PXE must be the first boot entry in your bioses. When the nodes will make theirs PXE requests, they will be detected. waiting for 4 node(s)... DHCP REQUEST FROM A NEW NODE : 00:03:47:23:45:f2 [ 1 / 4 ] DHCP REQUEST FROM A NEW NODE : 00:f0:27:2e:13:e1 [ 2 / 4 ] DHCP REQUEST FROM A NEW NODE : 00:50:da:dc:b6:88 [ 3 / 4 ] dhcp request from a registered node : 00:f0:27:2e:13:e1 DHCP REQUEST FROM A NEW NODE : 00:90:27:2e:11:03 [ 4 / 4 ] Moving dhcpd.conf in /root... Moving /etc/dhcpd.conf.add_nodes in /etc/dhcpd.conf Shutting down dhcpd: [ OK ] Starting dhcpd: [ OK ]