infrastructure VM (IVM) for our SmartOS sandbox environment. In part 2,
we move on to our services IVM, "serv1". To recap, "serv1" will host
a name server, web server, and time server for our SmartOS compute nodes
(CNs) and SmartOS VMs (SOSVMs). (SmartOS setup and configuration begins
in part 3). The VirtualBox (VBox) configuration for "serv1" is:
- 1 vcpu
- 1024 MB memory
- 8 GB vdisk (dynamic / thin provisioned)
- 2 configured interfaces in VBox:
+ Adapter 1: internal network (intnet (10.0.7.10))
+ Adapter 2: internal network (labworld (10.0.8.10))
During initial installation of CentOS 6.2, I chose a "webserver"install and added packages for "bind" and "ntpd" to host our name and
time servers, respectively. Following the installation, we end up with
the following system configuration:
serv1 [0] /usr/bin/awk '{print $0 ORS}' RS='' /etc/sysconfig/network-scripts/ifcfg-eth*
DEVICE="eth0"
NM_CONTROLLED="no"
ONBOOT=yes
HWADDR=08:00:27:49:3F:CA
TYPE=Ethernet
BOOTPROTO=none
IPADDR=10.0.7.10
NETMASK=255.255.255.0
BROADCAST=10.0.7.255
IPV6INIT=no
DEVICE="eth1"
NM_CONTROLLED="no"
ONBOOT=yes
HWADDR=08:00:27:7D:54:DC
TYPE=Ethernet
BOOTPROTO=none
IPADDR=10.0.8.10
NETMASK=255.255.255.0
BROADCAST=10.0.8.255
IPV6INIT=no
serv1 [0] /sbin/ifconfig -a
eth0 Link encap:Ethernet HWaddr 08:00:27:49:3F:CA
inet addr:10.0.7.10 Bcast:10.0.7.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe49:3fca/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:603 errors:0 dropped:0 overruns:0 frame:0
TX packets:359 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:56411 (55.0 KiB) TX bytes:49299 (48.1 KiB)
eth1 Link encap:Ethernet HWaddr 08:00:27:7D:54:DC
inet addr:10.0.8.10 Bcast:10.0.8.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe7d:54dc/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:7 errors:0 dropped:0 overruns:0 frame:0
TX packets:21 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:466 (466.0 b) TX bytes:1386 (1.3 KiB)
serv1 [0] /bin/cat /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=serv1.admin.none
GATEWAY=10.0.7.37
NOZEROCONF=yes
serv1 [0] /bin/netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
10.0.7.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.0.8.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
0.0.0.0 10.0.7.37 0.0.0.0 UG 0 0 0 eth0
serv1 [0] /bin/cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.7.10 serv1.admin.none serv1
serv1 [0] /bin/cat /etc/resolv.conf
search vbox.none admin.none world.none
nameserver 10.0.7.10
domain admin.none
serv1 [0]
Next, since this is a wholly self-contained sandbox environment withno external access, we can be somewhat lax in security. As such, I've
disabled 'iptables' and 'selinux':
serv1 [0] /sbin/chkconfig --level 2345 iptables off
serv1 [0] /sbin/chkconfig --level 2345 ip6tables off
serv1 [0] /sbin/iptables --flush
serv1 [0] /bin/egrep -v '^#|^$' /etc/selinux/config
SELINUX=disabled
SELINUXTYPE=targeted
serv1 [0]
On to our first infrastructure service, the time server. With thesandbox entirely private, the following lines in "/etc/ntp.conf" need
to be uncommented:
server 127.127.1.0 # local clock
fudge 127.127.1.0 stratum 10
These effectively configure 'ntpd' to assume the local time on the host(serv1) is always correct. Additionally, the following lines were added
for both the internal admin network (10.0.7.0/24) and the world network
(10.0.8.0/24):
restrict 10.0.7.0 mask 255.255.255.0 nomodify notrap
restrict 10.0.8.0 mask 255.255.255.0 nomodify notrap
The resulting "ntp.conf" file is:
serv1 [0] /bin/egrep -v '^#|^$' /etc/ntp.conf
driftfile /var/lib/ntp/drift
restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery
restrict 127.0.0.1
restrict -6 ::1
restrict 10.0.7.0 mask 255.255.255.0 nomodify notrap
restrict 10.0.8.0 mask 255.255.255.0 nomodify notrap
server 127.127.1.0 # local clock
fudge 127.127.1.0 stratum 10
includefile /etc/ntp/crypto/pw
keys /etc/ntp/keys
With 'ntpd' configured, we can now enable it under runlevel 3 andstart it:
serv1 [0] /sbin/runlevel
N 3
serv1 [0] /sbin/chkconfig --level 3 ntpd on
serv1 [0] /sbin/service ntpd start
Starting ntpd: [ OK ]
serv1 [0]
The second service to prepare for is our web server (SmartOS imageserver). For this, I downloaded the default page returned from:
https://datasets.joyent.com/datasets/
and saved it as "~/joyent/datasets/index.html" on "glados" (myworkstation). The "index.html" contents are actually JSON objects
detailing the dsmanifest files for each image as explained here.
Since I only want a subset of those images available, I've modified the
contents of the "index.html" file to retain only those I want. Next,
since each image object details the downloadable image URL from Joyent,
I downloaded those images (that I've kept in the updated "index.html")
to their respective UUID named image directories. After creating
"/var/www/joyent/datasets" on "serv1", I simply copied over the resulting
"index.html" and dataset images to "serv1" via 'scp':
# on serv1:
serv1 [0] /bin/mkdir -p /var/www/joyent/datasets
# on glados:
troy@glados [0] /usr/bin/scp -r ~/joyent/datasets/* root@serv1:/var/www/joyent/datasets/.
root@serv1's password:
base64-1.8.4.zfs.bz2 100% 78MB 15.5MB/s 00:05
index.html 100% 10KB 9.9KB/s 00:00
<snip...>
nodejs-1.4.0.zfs.bz2 20% 54MB 27.2MB/s 00:07 ETA
troy@glados [0]
Back on "serv1", we can see our resulting image repository file /directory structure:
serv1 [0] /usr/bin/find /var/www/joyent -print
/var/www/joyent
/var/www/joyent/datasets
/var/www/joyent/datasets/index.html
/var/www/joyent/datasets/b00acc20-14ab-11e2-85ae-4f03a066e93e
/var/www/joyent/datasets/b00acc20-14ab-11e2-85ae-4f03a066e93e/mongodb-1.4.0.zfs.bz2
/var/www/joyent/datasets/3390ca7c-f2e7-11e1-8818-c36e0b12e58b
/var/www/joyent/datasets/3390ca7c-f2e7-11e1-8818-c36e0b12e58b/standard-1.0.7.zfs.bz2
/var/www/joyent/datasets/ef22b182-3d7a-11e2-a7a9-af27913943e2
/var/www/joyent/datasets/ef22b182-3d7a-11e2-a7a9-af27913943e2/base-1.8.2.zfs.bz2
/var/www/joyent/datasets/dc1a8b5e-043c-11e2-9d94-0f3fcb2b0c6d
/var/www/joyent/datasets/dc1a8b5e-043c-11e2-9d94-0f3fcb2b0c6d/percona-1.6.0.zfs.bz2
/var/www/joyent/datasets/84cb7edc-3f22-11e2-8a2a-3f2a7b148699
/var/www/joyent/datasets/84cb7edc-3f22-11e2-8a2a-3f2a7b148699/base-1.8.4.zfs.bz2
/var/www/joyent/datasets/fdea06b0-3f24-11e2-ac50-0b645575ce9d
/var/www/joyent/datasets/fdea06b0-3f24-11e2-ac50-0b645575ce9d/base64-1.8.4.zfs.bz2
/var/www/joyent/datasets/aa583f78-3d83-11e2-9188-fff9b605718d
/var/www/joyent/datasets/aa583f78-3d83-11e2-9188-fff9b605718d/base64-1.8.2.zfs.bz2
/var/www/joyent/datasets/1fc068b0-13b0-11e2-9f4e-2f3f6a96d9bc
/var/www/joyent/datasets/1fc068b0-13b0-11e2-9f4e-2f3f6a96d9bc/nodejs-1.4.0.zfs.bz2
/var/www/joyent/datasets/a0f8cf30-f2ea-11e1-8a51-5793736be67c
/var/www/joyent/datasets/a0f8cf30-f2ea-11e1-8a51-5793736be67c/standard64-1.0.7.zfs.bz2
serv1 [0]
Before our image repository can be of any use, we need to further modifyour "index.html" file to set the image URL to be relative to our image
server IP address (10.0.7.10) and configuration on "serv1":
serv1 [0] /usr/bin/head -20 /var/www/joyent/datasets/index.html
[
{
"name": "base64",
"version": "1.8.4",
"type": "zone-dataset",
"description": "Base image with core packages preinstalled",
"published_at": "2012-12-05T21:59:37.513Z",
"os": "smartos",
"files": [
{
"path": "base64-1.8.4.zfs.bz2",
"sha1": "da918c38f1484b4cfc2c5e15b9b96583fae44b1c",
"size": 81369208,
"url": "http://10.0.7.10/datasets/fdea06b0-3f24-11e2-ac50-0b645575ce9d/base64-1.8.4.zfs.bz2"
}
],
"requirements": {
"networks": [
{
"name": "net0",
<snip...>
(The resulting "index.html" file can be downloaded from here.) Next, I'veadded an apache include file (joyent.conf) to present our image server.
Following this, I've also enabled 'httpd' so it starts after we reboot
"serv1" later:
serv1 [0] /bin/cat /etc/httpd/conf.d/joyent.conf
<Directory "/var/www/joyent">
Options Indexes
AllowOverride None
Order allow,deny
Allow from all
</Directory>
<VirtualHost 10.0.7.10:80>
DocumentRoot /var/www/joyent
ServerName serv1.admin.none
ErrorLog logs/serv1-error_log
CustomLog logs/serv1-access_log common
</VirtualHost>
serv1 [0] /sbin/chkconfig --list | /bin/grep http
httpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
serv1 [0] /sbin/chkconfig --level 3 httpd on
serv1 [0]
(Since "serv1" is its own name server, the FQDN for "ServerName" in the"VirtualHost" configuration above is also detailed in "/etc/hosts" on
"serv1" in case 'httpd' starts before 'named'.) Our third, and final
service to configure is 'bind' (named). Below, I've set the system
configuration file for 'named' (for start up purposes) and listed out
all of our 'named' configuration and zone files:
serv1 [0] /bin/grep -v ^# /etc/sysconfig/named
OPTIONS="-c /opt/bind-config/named.conf"
serv1 [0] /bin/ls -F /opt/bind-config
named.conf zones/
serv1 [0] find /opt/bind-config -print
/opt/bind-config
/opt/bind-config/named.conf
/opt/bind-config/zones
/opt/bind-config/zones/10.0.8.rev
/opt/bind-config/zones/vbox.none
/opt/bind-config/zones/localhost
/opt/bind-config/zones/10.0.7.rev
/opt/bind-config/zones/192.168.56.rev
/opt/bind-config/zones/127.0.0.rev
/opt/bind-config/zones/admin.none
/opt/bind-config/zones/world.none
serv1 [0]
As seen above, there is a zone file for each domain and network thatwe are using. Below, I've included the full contents of the 'named'
configuration and zone files:
serv1 [0] /bin/cat /opt/bind-config/named.conf
options {
directory "/opt/bind-config";
allow-transfer { 10.0.7.0/24; 192.168.56.0/24; };
allow-recursion { none; };
version "lab server 0.3.1";
};
# logging directives
logging {
channel named_log {
file "named.log" versions 3 size 25m;
severity info;
print-severity yes;
print-time yes;
print-category yes;
};
category default {
named_log;
};
};
zone "admin.none" {
type master;
file "zones/admin.none";
};
zone "7.0.10.in-addr.arpa" {
type master;
file "zones/10.0.7.rev";
};
zone "world.none" {
type master;
file "zones/world.none";
};
zone "8.0.10.in-addr.arpa" {
type master;
file "zones/10.0.8.rev";
};
zone "vbox.none" {
type master;
file "zones/vbox.none";
};
zone "56.168.192.in-addr.arpa" {
type master;
file "zones/192.168.56.rev";
};
zone "localhost" {
type master;
file "zones/localhost";
allow-update { none; };
};
zone "0.0.127.in-addr.arpa" {
type master;
file "zones/127.0.0.rev";
allow-update { none; };
};
serv1 [0] for i in admin.none 10.0.7.rev world.none 10.0.8.rev \
> vbox.none 192.168.56.rev localhost 127.0.0.rev ; do \
> echo "==> /opt/bind-config/zones/${i} <==" ; \
> /bin/cat /opt/bind-config/zones/${i} ; echo ; done
==> /opt/bind-config/zones/admin.none <==
$TTL 86400 ; Default TTL (1 day)
@ SOA serv1.admin.none. hostmaster.admin.none. (
2013012801 ; Serial number (YYYYMMDDNN)
3600 ; Refresh (1 hour)
300 ; Retry (5 minutes)
864000 ; Expire (10 days)
86400 ) ; Negative TTL (1 day)
NS serv1.admin.none.
serv1 A 10.0.7.10
muttley A 10.0.7.37
cn40 A 10.0.7.40
cn41 A 10.0.7.41
cn42 A 10.0.7.42
cn43 A 10.0.7.43
cn44 A 10.0.7.44
cn45 A 10.0.7.45
cn46 A 10.0.7.46
cn47 A 10.0.7.47
cn48 A 10.0.7.48
cn49 A 10.0.7.49
cn50 A 10.0.7.50
==> /opt/bind-config/zones/10.0.7.rev <==
$TTL 86400 ; Default TTL (1 day)
@ SOA serv1.admin.none. hostmaster.admin.none. (
2013012801 ; Serial number (YYYYMMDDNN)
3600 ; Refresh (1 hour)
300 ; Retry (5 minutes)
864000 ; Expire (10 days)
86400 ) ; Negative TTL (1 day)
NS serv1.admin.none.
10 PTR serv1.admin.none.
37 PTR muttley.admin.none.
40 PTR cn40.admin.none.
41 PTR cn41.admin.none.
42 PTR cn42.admin.none.
43 PTR cn43.admin.none.
44 PTR cn44.admin.none.
45 PTR cn45.admin.none.
46 PTR cn46.admin.none.
47 PTR cn47.admin.none.
48 PTR cn48.admin.none.
49 PTR cn49.admin.none.
50 PTR cn50.admin.none.
==> /opt/bind-config/zones/world.none <==
$TTL 86400 ; Default TTL (1 day)
@ SOA serv1.admin.none. hostmaster.admin.none. (
2013012801 ; Serial number (YYYYMMDDNN)
3600 ; Refresh (1 hour)
300 ; Retry (5 minutes)
864000 ; Expire (10 days)
86400 ) ; Negative TTL (1 day)
NS serv1.admin.none.
serv1 A 10.0.8.10
muttley A 10.0.8.37
vm100 A 10.0.8.100
vm101 A 10.0.8.101
vm102 A 10.0.8.102
vm103 A 10.0.8.103
vm104 A 10.0.8.104
vm105 A 10.0.8.105
vm106 A 10.0.8.106
vm107 A 10.0.8.107
vm108 A 10.0.8.108
vm109 A 10.0.8.109
vm110 A 10.0.8.110
vm111 A 10.0.8.111
vm112 A 10.0.8.112
vm113 A 10.0.8.113
vm114 A 10.0.8.114
vm115 A 10.0.8.115
vm116 A 10.0.8.116
vm117 A 10.0.8.117
vm118 A 10.0.8.118
vm119 A 10.0.8.119
vm120 A 10.0.8.120
vm121 A 10.0.8.121
vm122 A 10.0.8.122
vm123 A 10.0.8.123
vm124 A 10.0.8.124
vm125 A 10.0.8.125
vm126 A 10.0.8.126
vm127 A 10.0.8.127
vm128 A 10.0.8.128
vm129 A 10.0.8.129
vm130 A 10.0.8.130
==> /opt/bind-config/zones/10.0.8.rev <==
$TTL 86400 ; Default TTL (1 day)
@ SOA serv1.admin.none. hostmaster.admin.none. (
2013012801 ; Serial number (YYYYMMDDNN)
3600 ; Refresh (1 hour)
300 ; Retry (5 minutes)
864000 ; Expire (10 days)
86400 ) ; Negative TTL (1 day)
NS serv1.admin.none.
10 PTR serv1.world.none.
37 PTR muttley.world.none.
100 PTR vm100.world.none.
101 PTR vm101.world.none.
102 PTR vm102.world.none.
103 PTR vm103.world.none.
104 PTR vm104.world.none.
105 PTR vm105.world.none.
106 PTR vm106.world.none.
107 PTR vm107.world.none.
108 PTR vm108.world.none.
109 PTR vm109.world.none.
110 PTR vm110.world.none.
111 PTR vm111.world.none.
112 PTR vm112.world.none.
113 PTR vm113.world.none.
114 PTR vm114.world.none.
115 PTR vm115.world.none.
116 PTR vm116.world.none.
117 PTR vm117.world.none.
118 PTR vm118.world.none.
119 PTR vm119.world.none.
120 PTR vm120.world.none.
121 PTR vm121.world.none.
122 PTR vm122.world.none.
123 PTR vm123.world.none.
124 PTR vm124.world.none.
125 PTR vm125.world.none.
126 PTR vm126.world.none.
127 PTR vm127.world.none.
128 PTR vm128.world.none.
129 PTR vm129.world.none.
130 PTR vm130.world.none.
==> /opt/bind-config/zones/vbox.none <==
$TTL 86400 ; Default TTL (1 day)
@ SOA serv1.admin.none. hostmaster.admin.none. (
2013012801 ; Serial number (YYYYMMDDNN)
3600 ; Refresh (1 hour)
300 ; Retry (5 minutes)
864000 ; Expire (10 days)
86400 ) ; Negative TTL (1 day)
NS serv1.admin.none.
glados A 192.168.56.1
muttley A 192.168.56.37
==> /opt/bind-config/zones/192.168.56.rev <==
$TTL 86400 ; Default TTL (1 day)
@ SOA serv1.admin.none. hostmaster.admin.none. (
2013012801 ; Serial number (YYYYMMDDNN)
3600 ; Refresh (1 hour)
300 ; Retry (5 minutes)
864000 ; Expire (10 days)
86400 ) ; Negative TTL (1 day)
NS serv1.admin.none.
1 PTR glados.vbox.none.
37 PTR muttley.vbox.none.
==> /opt/bind-config/zones/localhost <==
$TTL 86400 ; Default TTL (1 day)
@ SOA serv1.admin.none. hostmaster.admin.none. (
2013012801 ; Serial number (YYYYMMDDNN)
3600 ; Refresh (1 hour)
300 ; Retry (5 minutes)
864000 ; Expire (10 days)
86400 ) ; Negative TTL (1 day)
NS serv1.admin.none.
@ A 127.0.0.1
==> /opt/bind-config/zones/127.0.0.rev <==
$TTL 86400 ; Default TTL (1 day)
@ SOA serv1.admin.none. hostmaster.admin.none. (
2013012801 ; Serial number (YYYYMMDDNN)
3600 ; Refresh (1 hour)
300 ; Retry (5 minutes)
864000 ; Expire (10 days)
86400 ) ; Negative TTL (1 day)
NS serv1.admin.none.
1 PTR localhost.
With 'named' now configured, 'named' is enabled for runlevel 3 andfinally, we reboot "serv1":
serv1 [0] /sbin/chkconfig --level 3 named on
serv1 [0] /sbin/init 6
Once "serv1" reboots, it is ready to function as our services host.In the next section, part 3, we get to start checking out SmartOS and
our first compute node, "cn40".
see also:
Intro SmartOS Setup pt 1
Intro SmartOS Setup pt 3
Intro SmartOS Setup pt 4
SmartOS - HowImageServerWork
index.html for the Image Server in this write up
