02 February 2013

Intro SmartOS Setup pt 1

For a while now, I've wanted to check out Joyent's SmartOS but until
recently, simply didn't have the time. This series of posts details my
initial setup and testing of SmartOS. (To jump straight to SmartOS within
VirtualBox (VBox), see part 3.) For better or worse, I really enjoy
sandbox environments which usually have limited, if any, external access
allowing me greater control over my test environments. With that in
mind, I've set up a self-contained test environment (no external access)
within VBox with 3 "infrastructure" VMs (IVMs) to somewhat present a
stable infrastructure to test SmartOS. Unfortunately, the physical host
on which I'm running VBox is using an AMD chipset, so I can still test
SmartOS handling of OS virtualization but not KVM level virtualization.

Starting things off, my 3 IVMs are:
        - muttley (FreeBSD 8.2 router)
                + simple configuration to handle routing between our
                  3 networks:
                        = vboxnet (
                        = intnet ( ("admin" network))
                        = labworld ( ("world" network))
        - serv1 (CentOS 6.2 services host)
                + initial services are name server (bind), web server
                  (httpd), and time server (ntpd)
                + sits on both "intnet" and "labworld"
        - cn40 (SmartOS compute node)
                + hosts our SmartOS created VMs
                        = VMs will only sit on "labworld"
                + sits on both "intnet" and "labworld"; only directly
                  accessible via "intnet"
Technically speaking, there is also a 4th host, the one running VBox.
The only bits that really matter for this host are:
        - hostname:  glados
        - actually does have real world access
        - VBox virtual NIC (vNIC):  vboxnet0 (host-only network)
        - IP address:
        - network routes for:
                + gateway
                + gateway
Since "glados" is my everyday workstation, rather than reconfigure
name resolution, I simply added entries to "/etc/hosts" for "muttley",
"serv1", "cn40" (and several other compute nodes), and several SmartOS
VMs (SOSVMs). Used throughout these posts are also 3 domains:
        - vbox.none
                + only contains DNS entries for "muttley" and "glados"
                + related network (
        - admin.none
                + contains DNS entries for all compute nodes and the
                  services host
                + related network (
        - world.none
                + contains DNS entires for all "world" facing hosts / SOSVMs
                + related network (
The above domains and their related network zones are maintained by
"serv1" (see part 2). For "muttley" and "serv1", I simply chose OSes
that I already had on hand and was familiar with, you can choose to use
whatever you'd like.

Getting to the IVM setups, the VBox configuration for "muttley" is:
        - 1 vcpu
        - 1024 MB memory
        - 8 GB virtual disk (dynamic / thin provisioned)
        - 3 configured network interfaces:
                + Adapter 1:  Internal network (intnet (
                + Adapter 2:  Host-only (vboxnet0 (
                + Adapter 3:  Internal network (labworld (
After installing "muttley" with a generic FreeBSD 8.2 install, very little
actually needs to be configured. In fact, here is the configuration in
its entirety:
        muttley [0] /bin/cat /etc/rc.conf
        ifconfig_em0="inet netmask"
        ifconfig_em1="inet netmask"
        ifconfig_em2="inet netmask"

        muttley [0] /usr/bin/grep -v ^# /etc/hosts
        ::1     localhost       localhost    glados.vbox.none glados   muttley.vbox.none muttley
        muttley [0] cat /etc/resolv.conf
        domain vbox.none
        search vbox.none world.none admin.none
        muttley [0]
The key part to "/etc/rc.conf" is enabling IP forwarding via:
Also, the "defaultrouter" entry actually points to a non-existent host.
Of note, while not yet configured until part 2, I've configured all host
resolution to be done against "serv1" ( At this point, the
router IVM for our sandbox is now configured and needs to be rebooted.
In part 2, I'll detail the configuration for "serv1", our services host.

see also:
    Intro SmartOS Setup pt 2
    Intro SmartOS Setup pt 3
    Intro SmartOS Setup pt 4