20 February 2011

Configuring NFS in Solaris

Networked environments allow for the sharing of many resources, including
filesystems (FS).  As a long time industry standard in UNIX, NFS (network
file system) provides access to remote filesystems which appear similar
to local resources on client hosts.  Since Sun Microsystems originally
devised NFS, the following focuses on simple NFS server and client
configuration in Solaris (see note 1).  Our host details are:
        HOST (server):          snorkle (10.0.23.191)
        HOST (client):          solex   (10.0.23.161)
        PROMPT (root):          HOST [0]
        PROMPT (user):          troy@solex [0]
        USER UID:GID:           1000:1000 (on both server and client)
        OS:                     Solaris 10
        NOTE:                   Aside from the use of SMF, the following
                                should equally apply to previous versions
                                of Solaris as well.
Starting off with our server side, NFS requires at least 4 services
running (for sane usage), though possibly up to 8, depending on NFS
version and features (see note 2):
        svc:/network/rpc/bind:default           (required)
        svc:/network/nfs/status:default         (required)
        svc:/network/nfs/nlockmgr:default       (required)
        svc:/network/nfs/rquota:default         (optional)
        svc:/network/nfs/server:default         (required)
        svc:/network/nfs/mapid:default          (NFSv4, required)
        svc:/network/rpc/gss:default            (NFSv4, optional)
        /usr/lib/nfs/nfslogd                    (NFSv2, NFSv3, optional)
(While NFSv3 is detailed herein, NFSv4 configuration and usage isn't much
different.)  Below, we check which services are running and start those
that aren't:
        snorkle [0] /usr/bin/svcs -v svc:/network/rpc/bind:default
        STATE          NSTATE        STIME    CTID   FMRI
        online         -              0:32:19     44 svc:/network/rpc/bind:default
        snorkle [0] /usr/bin/svcs -v svc:/network/nfs/status:default \ 
        > svc:/network/nfs/nlockmgr:default svc:/network/nfs/rquota:default \
        > svc:/network/nfs/server:default 
        STATE          NSTATE        STIME    CTID   FMRI
        disabled       -              0:30:32      - svc:/network/nfs/status:default
        offline        -              1:39:00      - svc:/network/nfs/nlockmgr:default
        disabled       -              0:32:40      - svc:/network/nfs/rquota:default
        disabled       -              0:30:42      - svc:/network/nfs/server:default
        snorkle [0] for i in status nlockmgr rquota server ; do 
        > /usr/sbin/svcadm enable svc:/network/nfs/${i}:default
        > done
    (We could have alternately enabled the necessary NFS services via
    'svcadm enable -r svc:/network/nfs/server:default'.)
        snorkle [0] /usr/bin/svcs -v svc:/network/nfs/status:default \
        > svc:/network/nfs/nlockmgr:default svc:/network/nfs/rquota:default \
        > svc:/network/nfs/server:default
        STATE          NSTATE        STIME    CTID   FMRI
        online         -              1:46:25     81 svc:/network/nfs/status:default
        online         -              1:46:26     82 svc:/network/nfs/nlockmgr:default
        online         -              1:46:27      - svc:/network/nfs/rquota:default
        disabled       -              1:46:31      - svc:/network/nfs/server:default
After a recheck of the services, 'svc:/network/nfs/server:default'
is still disabled.  A check of the problem via 'svcs' and the SMF log
aren't exactly helpful either:
        snorkle [0] /usr/bin/svcs -xv svc:/network/nfs/server:default
        svc:/network/nfs/server:default (NFS server)
         State: disabled since February 20, 2011  1:46:31 AM EST
        Reason: Disabled by an administrator.
           See: http://sun.com/msg/SMF-8000-05
           See: man -M /usr/share/man -s 1M nfsd
           See: /var/svc/log/network-nfs-server:default.log
        Impact: This service is not running.
        snorkle [0] /usr/bin/cat /var/svc/log/network-nfs-server:default.log
        [ Sep  1 19:55:44 Disabled. ]
        [ Sep  1 19:55:44 Rereading configuration. ]
        [ Feb 20 01:46:25 Enabled. ]
        [ Feb 20 01:46:27 Executing start method ("/lib/svc/method/nfs-server start") ]
        [ Feb 20 01:46:27 Method "start" exited with status 0 ]
        [ Feb 20 01:46:27 Stopping because all processes in service exited. ]
        [ Feb 20 01:46:28 Executing stop method ("/lib/svc/method/nfs-server stop 83") ]
        [ Feb 20 01:46:28 Method "stop" exited with status 0 ]
        [ Feb 20 01:46:28 Executing start method ("/lib/svc/method/nfs-server start") ]
        [ Feb 20 01:46:29 Method "start" exited with status 0 ]
        [ Feb 20 01:46:29 Stopping because all processes in service exited. ]
        [ Feb 20 01:46:29 Executing stop method ("/lib/svc/method/nfs-server stop 85") ]
        [ Feb 20 01:46:29 Method "stop" exited with status 0 ]
        [ Feb 20 01:46:29 Executing start method ("/lib/svc/method/nfs-server start") ]
        [ Feb 20 01:46:30 Method "start" exited with status 0 ]
        [ Feb 20 01:46:30 Stopping because all processes in service exited. ]
        [ Feb 20 01:46:30 Executing stop method ("/lib/svc/method/nfs-server stop 87") ]
        [ Feb 20 01:46:31 Method "stop" exited with status 0 ]
        [ Feb 20 01:46:31 Disabled. ]
The reason that 'svc:/network/nfs/server:default' is still disabled is
because '/etc/dfs/dfstab' contains no entries of filesystems to share.
Below, we've updated 'dfstab' to include two shared filesystems (see
note 3):
        snorkle [0] /usr/bin/egrep -v '^$|^#' /etc/dfs/dfstab
        share -F nfs -o nosuid,rw=@10.0.22.0/23,anon=60001 -d "home dirs" /export/home
        share -F nfs -o rw=beastie:@10.0.23.191,root=@10.0.23.191,ro,nosub /usr/sfw
    As an aside, you can use entries from 'dfstab' directly on the
    command line.  This can be sometimes useful when you are initially
    configuring a share and need to work out the appropriate export
    options.  You can also stop sharing an exported FS via 'unshare':
        snorkle [0] /usr/sbin/share -F nfs -o ro=@10.0.22.0/23 /usr/share/man
        snorkle [0] /usr/sbin/unshare /usr/share/man
With 'dfstab' updated, 'svc:/network/nfs/server:default' will now start:
        snorkle [0] /usr/sbin/svcadm enable svc:/network/nfs/server:default
        snorkle [0] /usr/bin/svcs -v svc:/network/nfs/server:default
        STATE          NSTATE        STIME    CTID   FMRI
        online         -              2:56:43     91 svc:/network/nfs/server:default
To verify our exported filesystems, we can use 'dfshares', 'share',
or review the contents of 'sharetab':
        snorkle [0] /usr/sbin/dfshares
        RESOURCE                                  SERVER ACCESS    TRANSPORT
           snorkle:/usr/sfw                      snorkle  -         -
           snorkle:/export/home                  snorkle  -         -
        snorkle [0] /usr/sbin/share
        -               /usr/sfw   rw=beastie:@10.0.23.191,root=@10.0.23.191,ro,nosub   ""
        -               /export/home   nosuid,rw=@10.0.22.0/23,anon=60001   "home dirs"
        snorkle [0] /usr/bin/cat /etc/dfs/sharetab
        /usr/sfw        -       nfs     rw=beastie:@10.0.23.191,root=@10.0.23.191,ro,nosub
        /export/home    -       nfs     nosuid,rw=@10.0.22.0/23,anon=60001      home dirs
Now that the server is setup, we can work on the client host.  A Solaris
NFS client requires at least 4 services running (for sane usage), though
possibly 6 depending on NFS version (see note 2):
        svc:/network/rpc/bind:default           (required)
        svc:/network/nfs/status:default         (required)
        svc:/network/nfs/nlockmgr:default       (required)
        svc:/network/nfs/client:default         (required)
        svc:/network/nfs/cbd:default            (NFSv4, required)
        svc:/network/nfs/mapid:default          (NFSv4, required)
Below, we check the status of the services, starting them as needed, and
use 'dfshares' to review what shares are available from the NFS server:
        solex [0] /usr/bin/svcs -v svc:/network/nfs/client:default svc:/network/nfs/status:default \
        > svc:/network/nfs/nlockmgr:default svc:/network/rpc/bind:default
        STATE          NSTATE        STIME    CTID   FMRI
        disabled       -              3:07:42      - svc:/network/rpc/bind:default
        disabled       -              3:07:44      - svc:/network/nfs/status:default
        disabled       -              3:07:44      - svc:/network/nfs/nlockmgr:default
        disabled       -              3:07:45      - svc:/network/nfs/client:default
        solex [0] for i in rpc/bind nfs/status nfs/nlockmgr nfs/client ; do 
        > /usr/sbin/svcadm enable svc:/network/${i}:default ; done
        solex [0] /usr/bin/svcs -v svc:/network/nfs/client:default svc:/network/nfs/status:default \
        > svc:/network/nfs/nlockmgr:default svc:/network/rpc/bind:default
        STATE          NSTATE        STIME    CTID   FMRI
        online         -              3:25:48     86 svc:/network/nfs/status:default
        online         -              3:25:49     85 svc:/network/rpc/bind:default
        online         -              3:25:49     87 svc:/network/nfs/nlockmgr:default
        online         -              3:25:49      - svc:/network/nfs/client:default
        solex [0] /usr/sbin/dfshares 10.0.23.191
        RESOURCE                                  SERVER ACCESS    TRANSPORT
        10.0.23.191:/usr/sfw                  10.0.23.191  -         -
        10.0.23.191:/export/home              10.0.23.191  -         -
In Solaris, tradition has /export/home set up as the location of user
home directories with the intention that those home directories are
remounted to /home via NFS and AutoFS (automountd).  Since we are only
concerned with NFS at this time, we'll skip /home and create '/home2'
on the client host.  Following that, we mount '/export/home' from the
NFS server (10.0.23.191) to '/home2' and verify the mount:
        solex [0] /usr/bin/mkdir /home2
        solex [0] /usr/bin/ls -ld /home2         
        drwxr-xr-x   2 root     root           2 Feb 20 03:12 /home2/
        solex [0] /usr/sbin/mount -F nfs -o rw,bg,intr 10.0.23.191:/export/home /home2
        solex [0] /usr/bin/ls -ld /home2                            
        drwxr-xr-x   4 root     root         512 Dec 21 02:21 /home2/
        solex [0] /usr/sbin/df -h /home2
        Filesystem             size   used  avail capacity  Mounted on
        10.0.23.191:/export/home
                               7.9G   4.4G   3.4G    57%    /home2
        solex [0] /usr/sbin/mount | /usr/bin/grep /home2
        /home2 on 10.0.23.191:/export/home remote/read/write/setuid/devices/rstchown/bg/intr/xattr/dev=8740001 on Sun Feb 20 03:26:37 2011
It's notable that the timestamp on '/home2' changes from its original
modification time to the last modification time of '/export/home' on the
NFS server after we mount the share.  On 'solex' as user 'troy', we switch
to '/home2/troy' (10.0.23.191:/export/home/troy) and test out our access:
        troy@solex [0] cd /home2/troy
        troy@solex [0] echo "this is my file" >> myfile
        troy@solex [0] /usr/bin/cat myfile
        this is my file
        troy@solex [0] /usr/bin/ls -l myfile
        -rw-r--r--   1 troy     sysvuser      16 Feb 20 03:32 myfile
Alright, we can access and write to the shared filesystem.  Below,
we try to make 'myfile' setuid and executable.  Since '/export/home'
was deliberately exported 'nosuid', the NFS server allows us to set
'myfile' executable but silently ignores the setuid bit:
        troy@solex [0] /usr/bin/chmod 4755 myfile
        troy@solex [0] /usr/bin/ls -l myfile
        -rwxr-xr-x   1 troy     sysvuser      16 Feb 20 03:32 myfile
        troy@solex [0] /usr/bin/rm myfile
        troy@solex [0] /usr/bin/ls -l myfile
        myfile: No such file or directory
        troy@solex [2]
Further testing the NFS server export options, on 'solex' we create
'/opt/sfw' and try to mount '10.0.23.191:/usr/sfw/bin' to '/opt/sfw'.
We are denied because of export option 'nosub' (see note 4):
        solex [0] /usr/bin/mkdir -p /opt/sfw
        solex [0] /usr/sbin/mount -F nfs -o rw,intr,vers=3 10.0.23.191:/usr/sfw/bin /opt/sfw
        nfs mount: 10.0.23.191:/usr/sfw/bin: Permission denied
A subsequent mount of the exported FS to '/opt/sfw' succeeds and is verified:
        solex [33] /usr/sbin/mount -F nfs -o rw,intr,vers=3 10.0.23.191:/usr/sfw /opt/sfw
        solex [0] /usr/sbin/mount  | /usr/bin/grep /opt/sfw
        /opt/sfw on 10.0.23.191:/usr/sfw remote/read/write/setuid/devices/rstchown/intr/vers=3/xattr/dev=8740005 on Sun Feb 20 03:47:32 2011
        solex [0] /usr/sbin/df -h /opt/sfw
        Filesystem             size   used  avail capacity  Mounted on
        10.0.23.191:/usr/sfw   7.9G   4.4G   3.4G    57%    /opt/sfw
        solex [0] /usr/bin/ls /opt/sfw
        bin                   info                  mysql                 swat
        doc                   lib                   sbin                  troy
        i386-sun-solaris2.10  libexec               share
        include               man                   src
Again as user 'troy', on 'solex' we try to create another
file (also-mine), this time to the read-only exported FS
10.0.23.191:/usr/sfw (mounted at /opt/sfw):
        troy@solex [0] /usr/bin/ls -ld /opt/sfw/troy
        drwxr-xr-x   2 troy     sysvuser     512 Feb 20 03:49 /opt/sfw/troy
        troy@solex [0] cd /opt/sfw/troy
        troy@solex [0] echo "this is also my file" >> also-mine
        -ksh: also-mine: cannot create [Read-only file system]
        solex [0] /usr/sbin/umount /opt/sfw
        solex [0] /usr/sbin/umount /home2
The above is to illustrate that export options (ro) from the NFS
server take precedence over the 'mount' options (rw) used by the client.
After the "Read-only" error, we've unmounted both '/opt/sfw' and '/home2'.
Rather than manually mounting an NFS share each time the host reboots,
I've added an entry to '/etc/vfstab' on the last line below for '/home2':
        solex [0] /usr/bin/cat /etc/vfstab
        #device         device          mount           FS      fsck    mount   mount
        #to mount       to fsck         point           type    pass    at boot options
        #
        /devices        -               /devices        devfs   -       no      -
        /proc           -               /proc           proc    -       no      -
        ctfs            -               /system/contract ctfs   -       no      -
        objfs           -               /system/object  objfs   -       no      -
        sharefs         -               /etc/dfs/sharetab       sharefs -       no      -
        fd              -               /dev/fd         fd      -       no      -
        swap            -               /tmp            tmpfs   -       yes     -
        /dev/zvol/dsk/rpool1/swap       -               -               swap    -       no      -
        10.0.23.191:/export/home        -       /home2  nfs     -       yes     rw,bg,intr
On the server, you can use 'unshareall' to stop sharing all exported
filesystems and verify with 'dfshares':
        snorkle [0] /usr/sbin/unshareall      
        snorkle [0] /usr/sbin/dfshares
        snorkle [1] 

NOTES

Note 1: The details provided herein do not take into account any potential
    security issues and assume access via a local LAN segment.

Note 2: Server / Client services:
    server
        svc:/network/rpc/bind:default           (rpcbind) converts rpc
                                                prog numbers into
                                                universal addresses 
                                                (must be running to use rpc)
        svc:/network/nfs/status:default         (statd) tracks clients
                                                holding file locks, informs
                                                server of client reboot so
                                                that locks can be released
        svc:/network/nfs/nlockmgr:default       (lockd) manages file locks
        svc:/network/nfs/mapid:default          (nfsmapid) v4, maps uids / gids
        svc:/network/nfs/server:default         (nfsd) also starts 'mountd'
        svc:/network/rpc/gss:default            (gssd) v4, gss-api protections
        svc:/network/nfs/rquota:default         (rquotad) nfs quota reporting
                                                daemon, optional
        /usr/lib/nfs/nfslogd                    logs NFS RPC activity for FS
                                                exported with logging enabled,
                                                not supported with v4, Each
                                                record in the log file includes
                                                a time stamp, the IP address (or
                                                hostname if it can be resolved)
                                                of the client system, the file
                                                or directory name the operation
                                                was performed on, and the type
                                                of operation. (optional)

    client
        svc:/network/rpc/bind:default           (rpcbind) converts rpc
                                                prog numbers into
                                                universal addresses
                                                (must be running to use rpc)
        svc:/network/nfs/status:default         (statd) tracks clients
                                                holding file locks, informs
                                                server of client reboot so
                                                that locks can be released
        svc:/network/nfs/nlockmgr:default       (lockd) manages file locks
        svc:/network/nfs/client:default         runs 'mountall -F nfs'
        svc:/network/nfs/cbd:default            (nfs4cbd) v4, callback daemon
        svc:/network/nfs/mapid:default          (nfsmapid) v4, maps uids / gids
Note 3: The breakdown of 'dfstab' entries reads:
    share -F nfs -o nosuid,rw=@10.0.22.0/23,anon=60001 -d "home dirs" /export/home
    share -F nfs -o rw=beastie:@10.0.23.191,root=@10.0.23.191,ro,nosub /usr/sfw

        share:                                  NFS share command (/usr/sbin/share)
        -F nfs                                  filesystem type to export
        -o nosuid,rw=@10.0.22.0/23,anon=60001   FS options
                nosuid                          client cannot create setuid executables
                                                (negates the default to allow setuid
                                                file creation)
                rw=@10.0.22.0/23                clients with IP addresses between
                                                10.0.22.0 - 10.0.23.255 have write access
                anon=60001                      unknown users will have an effective UID
                                                of 60001
                rw=beastie:@10.0.23.191         write access only available to host
                                                beastie and host 10.0.23.191
                root=@10.0.23.191               normally the root user
                                                on the client host
                                                accesses a share with
                                                permissions / UID set by
                                                anon, however 'root='
                                                specifies that root on
                                                host 10.0.23.191 retains
                                                root privileges / UID
                ro                              read only access to any host not previously
                                                defined by 'rw='
                nosub                           only the exported
                                                directory can be mounted
                                                by a client host,
                                                direct mounting of sub
                                                directories is disallowed
        -d "home dirs"                          FS description
        (/export/home|/usr/sfw)                 directory to be shared (exported)
Note 4: Because NFSv4 does not use the MOUNT protocol, 'nosub' only
    impacts client side mounting using NFSv2 and NFSv3.  Since Solaris 10
    attempts use of NFSv4 by default, falling back to v2 or v3 as necessary,
    to illustrate 'nosub' I deliberately set option 'vers=3' in the mount
    command:
        solex [33] /usr/sbin/mount -F nfs -o rw,intr,vers=3 10.0.23.191:/usr/sfw /opt/sfw

see also:
    Configuring NFS in Linux
    Configuring NFS in FreeBSD
    Configuring NFS in SmartOS