Manager (VxVM) with Sun Volume Manager (SVM) for control of the root disk
while leaving application volumes under VxVM control. I had originally
tested this against Solaris 9 sparc running VxVM 4.0, though have updated
it for Solaris 10 x86 running VxVM 5.0MP3. The commands are still the
same, with the addition of particulars for x86 and handling of Solaris
boot archives introduced in Solaris 10. Additional notes can be found
at the end. Sections are labeled 'COMMANDS (detailed)', 'COMMANDS
(quick)', and NOTES. Given the complexity involved and multitude of
steps, please read entirely through and understand both the 'COMMANDS
(detailed)' and 'NOTES' sections before attempting to perform this on
any host. Host details are:
HOST: snorkle (x86) SHELL PROMPT: snorkle [0] CDROM PROMPT: sol10cdrom [0] OS: Solaris 10 u8 x86 ROOT VOLUMES: rootvol swapvol var dm type plex dg PRIMARY DISK: c1t0d0s2 auto:sliced rootdg01 rootdg SECONDARY DISK: c1t1d0s2 auto:sliced rootmirror rootdg x86 BOOTLOADER: grubThe actual work necessary to accomplish this scenario can be see as two
separate parts, 1) removing VxVM control of the root disk, 2) setting
up SVM control of the root disk. As a disclaimer, part 1 has been
floating around on the net for a while, though typically just a listing
of commands without expected output. Part 2 should be mostly trivial for
anyone familiar with SVM. What follows simply brings the two together.
COMMANDS (detailed)
Begin by rebooting the host to single user on a boot cdrom. This will
allow us to make some necesary modifications to the primary root mirror
slice (c1t0d0s0):
snorkle [0] /usr/sbin/reboot -- 'cdrom -s'For x86, you will likely need to specify cdrom boot from the BIOS and
modify the grub boot option appropriately for boot to single user.
Once in single user, we need to check 'c1t0d0s0' for any issues, mount
it, and make backup copies of /etc/vfstab and /etc/system:
sol10cdrom [0] /usr/sbin/fsck -y /dev/rdsk/c1t0d0s0 ** /dev/rdsk/c1t0d0s0 ** Last Mounted on / ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3a - Check Connectivity ** Phase 3b - Verify Shadows/ACLs ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cylinder Groups 157286 files, 4379686 used, 3878825 free (5385 frags, 484180 blocks, 0.1% fragmentation) sol10cdrom [0] [ ! -d /a ] && /usr/bin/mkdir /a sol10cdrom [0] /usr/sbin/mount /dev/dsk/c1t0d0s0 /a sol10cdrom [0] /usr/bin/cp /a/etc/system /a/etc/system.orig sol10cdrom [0] /usr/bin/cp /a/etc/vfstab /a/etc/vfstab.origThe following lines in /a/etc/system need to be commented out or removed:
rootdev:/pseudo/vxio@0:0 set vxio:vol_rootdev_is_volume=1We can comment these line out via:
sol10cdrom [0] /usr/bin/sed -e 's;rootdev:/pseudo/vxio@0:0;*rootdev:/pseudo/vxio@0:0;' /a/etc/system.orig > /a/etc/system sol10cdrom [0] /usr/bin/cp /a/etc/system /a/etc/system.orig1 sol10cdrom [0] /usr/bin/sed -e 's;set vxio:vol_rootdev_is_volume=1;*set vxio:vol_rootdev_is_volume=1;' /a/etc/system.orig1 > /a/etc/systemNext, we need to update the current 'vfstab' to this:
sol10cdrom [0] /usr/bin/cat /a/etc/vfstab #device device mount FS fsck mount mount #to mount to fsck point type pass at boot options # fd - /dev/fd fd - no - /proc - /proc proc - no - #/dev/vx/dsk/bootdg/swapvol - - swap - no - #/dev/vx/dsk/bootdg/rootvol /dev/vx/rdsk/bootdg/rootvol / ufs 1 no - #/dev/vx/dsk/bootdg/var /dev/vx/rdsk/bootdg/var /var ufs 1 no - /dev/dsk/c1t0d0s1 - - swap - no - /dev/dsk/c1t0d0s0 /dev/rdsk/c1t0d0s0 / ufs 1 no - /dev/dsk/c1t0d0s3 /dev/rdsk/c1t0d0s3 /var ufs 1 no - /devices - /devices devfs - no - sharefs - /etc/dfs/sharetab sharefs - no - ctfs - /system/contract ctfs - no - objfs - /system/object objfs - no - swap - /tmp tmpfs - yes - #vshare - /mnt/vshare vboxfs - no - #/dev/vx/dsk/mydg/myvol /dev/vx/rdsk/mydg/myvol /mnt/myvol vxfs 1 no - #NOTE: volume rootvol (/) encapsulated partition c1t0d0s0 #NOTE: volume swapvol (swap) encapsulated partition c1t0d0s1 #NOTE: volume var (/var) encapsulated partition c1t0d0s3In the above, all VxVM volumes have been commented out and traditional
ctds values have been setup for the root volumes. After this, we need
prevent VxVM from starting at boot time and attempting to resync its
root mirrors. To do so, create file 'install-db' and remove 'root-done'
(which tells VxVM that the root disk is encapsulated). For Solaris 10
hosts, the boot archive must also be updated since we made changes to
'/a/etc/system'. Finally reboot to single user on the root disk (c1t0d0):
sol10cdrom [0] /usr/bin/touch /a/etc/vx/reconfig.d/state.d/install-db sol10cdrom [0] /usr/bin/rm /a/etc/vx/reconfig.d/state.d/root-done sol10cdrom [0] /usr/sbin/bootadm update-archive -R /a sol10cdrom [0] /usr/sbin/reboot -- -sAfter the reboot, we can see that the root disk is now configured with
ctds values rather than VxVM volumes:
snorkle [0] /usr/sbin/df -h Filesystem size used avail capacity Mounted on /dev/dsk/c1t0d0s0 7.9G 4.2G 3.6G 54% / /devices 0K 0K 0K 0% /devices ctfs 0K 0K 0K 0% /system/contract proc 0K 0K 0K 0% /proc mnttab 0K 0K 0K 0% /etc/mnttab swap 1.8G 932K 1.8G 1% /etc/svc/volatile objfs 0K 0K 0K 0% /system/object sharefs 0K 0K 0K 0% /etc/dfs/sharetab fd 0K 0K 0K 0% /dev/fd /dev/dsk/c1t0d0s3 1.1G 407M 633M 40% /var swap 1.8G 0K 1.8G 0% /tmp swap 1.8G 4K 1.8G 1% /var/runIt will be necessary to start VxVM manually so that the root volumes can
be removed and the root (c1t0d0) and mirror (c1t1d0) disks can be freed
from VxVM control. To do so:
snorkle [0] /usr/bin/rm /etc/vx/reconfig.d/state.d/install-db snorkle [0] /usr/sbin/vxiod set 10 snorkle [0] /usr/sbin/vxconfigd -m disable snorkle [0] /usr/sbin/vxdctl enable snorkle [0] /usr/sbin/vxdisk list DEVICE TYPE DISK GROUP STATUS c1t0d0s2 auto:sliced rootdg01 rootdg online c1t1d0s2 auto:sliced rootmirror rootdg online c1t2d0s2 auto:simple mydg01 mydg online c1t3d0s2 auto:simple mydg02 mydg onlineNow we can remove the root volumes from VxVM and remove the DG:
snorkle [0] /usr/sbin/vxedit -g rootdg -rf rm rootvol snorkle [0] /usr/sbin/vxedit -g rootdg -rf rm swapvol snorkle [0] /usr/sbin/vxedit -g rootdg -rf rm var snorkle [0] /usr/sbin/vxdg -g rootdg rmdisk rootmirror snorkle [0] /usr/sbin/vxdg -g rootdg rmdisk rootdg01 VxVM vxdg ERROR V-5-1-552 Disk rootdg01 is used by one or more subdisks. Use -k to remove device assignment. snorkle [16] /usr/sbin/vxdg destroy rootdg snorkle [0] /usr/sbin/vxdisk list DEVICE TYPE DISK GROUP STATUS c1t0d0s2 auto:sliced - - online c1t1d0s2 auto:sliced - - online c1t2d0s2 auto:simple mydg01 mydg online c1t3d0s2 auto:simple mydg02 mydg onlineVxVM gives up an error above as you cannot remove the last disk in a
DG without destroying the DG, which we do in the subsequent 'destroy'
command. Our root disk and mirror freed of VxVM. If other VxVM volumes
still exist and are to be retained, they can be started and mounted
to ensure they are still functioning. (Remember to uncomment them in
/etc/vfstab first):
snorkle [0] /usr/sbin/vxvol -g mydg startall snorkle [0] /usr/sbin/mount /mnt/myvol snorkle [0] /usr/bin/df -h | /usr/bin/grep vx /dev/vx/dsk/mydg/myvol 894M 8.1M 832M 1% /mnt/myvolThe final thing to do before getting into SVM is have VxVM restore the
original VTOC of 'c1t0d0'. The VTOC for 'ct10d0' is presented both
before and after the restoration:
snorkle [0] /usr/sbin/prtvtoc -h /dev/rdsk/c1t0d0s2 0 2 00 16065 16787925 16803989 / 1 3 01 16884315 2233035 19117349 2 5 00 0 21430710 21430709 3 7 00 19117350 2313360 21430709 /var 4 15 01 16803990 80325 16884314 7 14 01 16065 21414645 21430709 8 1 01 0 16065 16064 snorkle [0] /etc/vx/bin/vxedvtoc -f /etc/vx/reconfig.d/disk.d/c1t0d0/vtoc /dev/rdsk/c1t0d0s2 # THE ORIGINAL PARTITIONING IS AS FOLLOWS : #SLICE TAG FLAGS START SIZE 0 0x2 0x200 16065 16787925 1 0x3 0x201 16884315 2233035 2 0x5 0x200 0 21430710 3 0x7 0x200 19117350 2313360 4 0xf 0x201 16803990 80325 5 0x0 0x200 0 0 6 0x0 0x200 0 0 7 0xe 0x201 16065 21414645 8 0x1 0x201 0 16065 9 0x0 0x201 0 0 10 0x0 0x201 0 0 11 0x0 0x201 0 0 12 0x0 0x201 0 0 13 0x0 0x201 0 0 14 0x0 0x201 0 0 15 0x0 0x201 0 0 # THE NEW PARTITIONING WILL BE AS FOLLOWS : #SLICE TAG FLAGS START SIZE 0 0x2 0x200 16065 16787925 1 0x3 0x201 16803990 2313360 2 0x5 0x200 0 21430710 3 0x7 0x200 19117350 2313360 4 0x0 0x200 0 0 5 0x0 0x200 0 0 6 0x0 0x200 0 0 7 0x0 0x200 0 0 8 0x1 0x201 0 16065 9 0x0 0x201 0 0 10 0x0 0x201 0 0 11 0x0 0x201 0 0 12 0x0 0x201 0 0 13 0x0 0x201 0 0 14 0x0 0x201 0 0 15 0x0 0x201 0 0 DO YOU WANT TO WRITE THIS TO THE DISK ? [Y/N] :y WRITING THE NEW VTOC TO THE DISK snorkle [0] /usr/sbin/prtvtoc -h /dev/rdsk/c1t0d0s2 0 2 00 16065 16787925 16803989 / 1 3 01 16803990 2313360 19117349 2 5 00 0 21430710 21430709 3 7 00 19117350 2313360 21430709 /var 8 1 01 0 16065 16064As it stands, VxVM control of the root disk has been completed removed.
If this was our only intention, we could stop here and reboot to
multiuser. Since we are setting up SVM, we need to update /etc/system
so that SVM can run with only a single plex of a mirror:
snorkle [0] echo "set md:mirrored_root_flag=1" >> /etc/systemAdditionally, since this is an x86 host, we need to restore grub back
to its pre-VxVM state. When VxVM was setup, it created a backup of
the original menu.lst so we move it back into place. With this being
a Solaris 10 host, we also need to update the boot-archive prior to a
reboot into single user from the cdrom. To prevent VxVM from starting,
we also create file 'install-db' again:
snorkle [0] /usr/bin/mv /boot/grub/menu.lst /boot/grub/menu.lst.vxvm snorkle [0] /usr/bin/mv /boot/grub/menu.lst.pre_vxvm /boot/grub/menu.lst snorkle [0] /usr/sbin/bootadm update-archive -v changed /etc/system cannot find: /etc/cluster/nodeid: No such file or directory cannot find: /etc/devices/mdi_ib_cache: No such file or directory updating /platform/i86pc/boot_archive snorkle [0] /usr/bin/touch /etc/vx/reconfig.d/state.d/install-db snorkle [0] /usr/sbin/reboot -- 'cdrom -s'Remember, while the above 'reboot' command will suffice for sparc,
in x86 you may have to tell the BIOS to boot from cdrom and manage to
single user from there.
Now to the part that seems a little scary, using 'format' we will modify
the current VTOC of 'c1t0d0' (our root disk) to create space for SVM's
metadbs. To do this, like VxVM before us, we will steal the space from
'swap'. Below is the output from 'verify' in '/usr/sbin/format' on
'c1t0d0' prior to our modification:
Part Tag Flag Cylinders Size Blocks 0 root wm 1 - 1045 8.01GB (1045/0/0) 16787925 1 swap wu 1046 - 1189 1.10GB (144/0/0) 2313360 2 backup wm 0 - 1333 10.22GB (1334/0/0) 21430710 3 var wm 1190 - 1333 1.10GB (144/0/0) 2313360 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0 8 boot wu 0 - 0 7.84MB (1/0/0) 16065 9 unassigned wu 0 0 (0/0/0) 0The following is the partition table after modification of slices 1, 4,
and 6:
Part Tag Flag Cylinders Size Blocks 0 root wm 1 - 1045 8.01GB (1045/0/0) 16787925 1 swap wu 1050 - 1189 1.07GB (140/0/0) 2249100 2 backup wm 0 - 1333 10.22GB (1334/0/0) 21430710 3 var wm 1190 - 1333 1.10GB (144/0/0) 2313360 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 1046 - 1047 15.69MB (2/0/0) 32130 6 unassigned wm 1048 - 1049 15.69MB (2/0/0) 32130 7 unassigned wm 0 0 (0/0/0) 0 8 boot wu 0 - 0 7.84MB (1/0/0) 16065 9 unassigned wu 0 0 (0/0/0) 0In the above, we simply took 4 cylinders from the beginning of swap and
used them to setup slices 5 and 6. Special care should be taken while
doing the above as you otherwise risk serious data loss (see note 4).
After this, use 'fsck' to verify the root disk is sane and reboot back
into single user from the root disk:
sol10cdrom [0] /usr/sbin/fsck -n /dev/rdsk/c1t0d0s0 ** /dev/rdsk/c1t0d0s0 (NO WRITE) ** Last Mounted on / ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3a - Check Connectivity ** Phase 3b - Verify Shadow/ACLs ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cylinder Groups 157420 files, 4379707 used, 3878804 free (5404 frags, 484175 blocks, 0.1% fragmentation) sol10cdrom [0] /usr/sbin/reboot -- -sThe output below is simply verification that Solaris didn't freak out
on our change of the VTOC after the reboot:
snorkle [0] /usr/sbin/df -h Filesystem size used avail capacity Mounted on /dev/dsk/c1t0d0s0 7.9G 4.2G 3.6G 54% / /devices 0K 0K 0K 0% /devices ctfs 0K 0K 0K 0% /system/contract proc 0K 0K 0K 0% /proc mnttab 0K 0K 0K 0% /etc/mnttab swap 1.8G 932K 1.8G 1% /etc/svc/volatile objfs 0K 0K 0K 0% /system/object sharefs 0K 0K 0K 0% /etc/dfs/sharetab fd 0K 0K 0K 0% /dev/fd /dev/dsk/c1t0d0s3 1.1G 407M 633M 40% /var swap 1.8G 0K 1.8G 0% /tmp swap 1.8G 4K 1.8G 1% /var/run snorkle [0] /usr/sbin/prtvtoc -h /dev/rdsk/c1t0d0s2 0 2 00 16065 16787925 16803989 / 1 3 01 16868250 2249100 19117349 2 5 00 0 21430710 21430709 3 7 00 19117350 2313360 21430709 /var 5 0 00 16803990 32130 16836119 6 0 00 16836120 32130 16868249 8 1 01 0 16065 16064Now, update the partition table of c1t1d0 based upon that of c1t0d0,
create metadbs on both c1t0d0 and c1t1d0, and setup the initial SVM
configuration of the root disk:
snorkle [0] /usr/sbin/prtvtoc -h /dev/rdsk/c1t0d0s2 | /usr/sbin/fmthard -s - /dev/rdsk/c1t1d0s2 fmthard: New volume table of contents now in place. snorkle [0] /usr/sbin/prtvtoc -h /dev/rdsk/c1t1d0s2 0 2 00 16065 16787925 16803989 1 3 01 16868250 2249100 19117349 2 5 00 0 21430710 21430709 3 7 00 19117350 2313360 21430709 5 0 00 16803990 32130 16836119 6 0 00 16836120 32130 16868249 8 1 01 0 16065 16064 snorkle [0] /usr/sbin/metadb -a -c3 -f c1t0d0s5 c1t0d0s6 c1t1d0s5 c1t1d0s6 snorkle [0] /usr/sbin/metainit -f d0 1 1 c1t0d0s0 d0: Concat/Stripe is setup snorkle [0] /usr/sbin/metainit d2 -m d0 d2: Mirror is setup snorkle [0] /usr/sbin/metainit -f d3 1 1 c1t0d0s1 d3: Concat/Stripe is setup snorkle [0] /usr/sbin/metainit d5 -m d3 d5: Mirror is setup snorkle [0] /usr/sbin/metainit -f d6 1 1 c1t0d0s3 d6: Concat/Stripe is setup snorkle [0] /usr/sbin/metainit d8 -m d6 d8: Mirror is setup snorkle [0] /usr/sbin/metaroot d2We need to update the 'vfstab' from this:
snorkle [0] /usr/bin/cat /etc/vfstab #device device mount FS fsck mount mount #to mount to fsck point type pass at boot options # fd - /dev/fd fd - no - /proc - /proc proc - no - #/dev/vx/dsk/bootdg/swapvol - - swap - no - #/dev/vx/dsk/bootdg/rootvol /dev/vx/rdsk/bootdg/rootvol / ufs 1 no - #/dev/vx/dsk/bootdg/var /dev/vx/rdsk/bootdg/var /var ufs 1 no - /dev/dsk/c1t0d0s1 - - swap - no - /dev/md/dsk/d2 /dev/md/rdsk/d2 / ufs 1 no - /dev/dsk/c1t0d0s3 /dev/rdsk/c1t0d0s3 /var ufs 1 no - /devices - /devices devfs - no - sharefs - /etc/dfs/sharetab sharefs - no - ctfs - /system/contract ctfs - no - objfs - /system/object objfs - no - swap - /tmp tmpfs - yes - #vshare - /mnt/vshare vboxfs - no - /dev/vx/dsk/mydg/myvol /dev/vx/rdsk/mydg/myvol /mnt/myvol vxfs 1 no - #NOTE: volume rootvol (/) encapsulated partition c1t0d0s0 #NOTE: volume swapvol (swap) encapsulated partition c1t0d0s1 #NOTE: volume var (/var) encapsulated partition c1t0d0s3To this, removing the unnecessary VxVM root volumes and configuring the
other SVM devices:
snorkle [0] /usr/bin/cat /etc/vfstab #device device mount FS fsck mount mount #to mount to fsck point type pass at boot options # fd - /dev/fd fd - no - /proc - /proc proc - no - /dev/md/dsk/d5 - - swap - no - /dev/md/dsk/d2 /dev/md/rdsk/d2 / ufs 1 no - /dev/md/dsk/d8 /dev/md/rdsk/d8 /var ufs 1 no - /devices - /devices devfs - no - sharefs - /etc/dfs/sharetab sharefs - no - ctfs - /system/contract ctfs - no - objfs - /system/object objfs - no - swap - /tmp tmpfs - yes - #vshare - /mnt/vshare vboxfs - no - /dev/vx/dsk/mydg/myvol /dev/vx/rdsk/mydg/myvol /mnt/myvol vxfs 1 no - #NOTE: volume d2 (/) svm c1t[0|1]d0s0 #NOTE: volume d5 (swap) svm c1t[0|1]d0s1 #NOTE: volume d8 (/var) svm c1t[0|1]d0s3To allow VxVM to restart after reboot, remove 'install-db'. Again,
being this is Solaris 10, update the boot archive. Finally reboot
into multiuser:
snorkle [0] /usr/bin/rm /etc/vx/reconfig.d/state.d/install-db snorkle [0] /usr/sbin/bootadm update-archive -v changed /etc/system cannot find: /etc/cluster/nodeid: No such file or directory cannot find: /etc/devices/mdi_ib_cache: No such file or directory updating /platform/i86pc/boot_archive snorkle [0] /usr/sbin/rebootAfter the reboot, I've done some verification work to ensure that our
metadbs are good, that we are now booted off of SVM metadevices, and
our VxVM volume is online.
snorkle [0] /usr/sbin/metadb -i flags first blk block count a m p luo 16 8192 /dev/dsk/c1t0d0s5 a p luo 8208 8192 /dev/dsk/c1t0d0s5 a p luo 16400 8192 /dev/dsk/c1t0d0s5 a p luo 16 8192 /dev/dsk/c1t0d0s6 a p luo 8208 8192 /dev/dsk/c1t0d0s6 a p luo 16400 8192 /dev/dsk/c1t0d0s6 a p luo 16 8192 /dev/dsk/c1t1d0s5 a p luo 8208 8192 /dev/dsk/c1t1d0s5 a p luo 16400 8192 /dev/dsk/c1t1d0s5 a p luo 16 8192 /dev/dsk/c1t1d0s6 a p luo 8208 8192 /dev/dsk/c1t1d0s6 a p luo 16400 8192 /dev/dsk/c1t1d0s6 r - replica does not have device relocation information o - replica active prior to last mddb configuration change u - replica is up to date l - locator for this replica was read successfully c - replica's location was in /etc/lvm/mddb.cf p - replica's location was patched in kernel m - replica is master, this is replica selected as input W - replica has device write errors a - replica is active, commits are occurring to this replica M - replica had problem with master blocks D - replica had problem with data blocks F - replica had format problems S - replica is too small to hold current data base R - replica had device read errors snorkle [0] /usr/sbin/df -h Filesystem size used avail capacity Mounted on /dev/md/dsk/d2 7.9G 4.2G 3.6G 54% / /devices 0K 0K 0K 0% /devices ctfs 0K 0K 0K 0% /system/contract proc 0K 0K 0K 0% /proc mnttab 0K 0K 0K 0% /etc/mnttab swap 1.6G 1.0M 1.6G 1% /etc/svc/volatile objfs 0K 0K 0K 0% /system/object sharefs 0K 0K 0K 0% /etc/dfs/sharetab fd 0K 0K 0K 0% /dev/fd /dev/md/dsk/d8 1.1G 407M 633M 40% /var swap 1.6G 0K 1.6G 0% /tmp swap 1.6G 12K 1.6G 1% /var/run swap 1.6G 0K 1.6G 0% /dev/vx/dmp swap 1.6G 0K 1.6G 0% /dev/vx/rdmp /dev/vx/dsk/mydg/myvol 894M 8.1M 832M 1% /mnt/myvolThe only thing left to do for SVM is to add in 'c1t1d0' to the SVM
configuration for the root disk. The following will attach c1t1d0 to
the mirror and automatically synchronizing the mirror in the background:
snorkle [0] /usr/sbin/metainit d1 1 1 c1t1d0s0 d1: Concat/Stripe is setup snorkle [0] /usr/sbin/metattach d2 d1 d2: submirror d1 is attached snorkle [0] /usr/sbin/metainit d4 1 1 c1t1d0s1 d4: Concat/Stripe is setup snorkle [0] /usr/sbin/metattach d5 d4 d5: submirror d4 is attached snorkle [0] /usr/sbin/metainit d7 1 1 c1t1d0s3 d7: Concat/Stripe is setup snorkle [0] /usr/sbin/metattach d8 d7 d8: submirror d7 is attachedTo view the current sync progress, use 'metastat':
snorkle [0] /usr/sbin/metastat | /usr/bin/egrep '^d|State:|in progress' d8: Mirror State: Okay State: Resyncing Resync in progress: 31 % done d6: Submirror of d8 State: Okay d7: Submirror of d8 State: Resyncing d5: Mirror State: Okay State: Resyncing Resync in progress: 44 % done d3: Submirror of d5 State: Okay d4: Submirror of d5 State: Resyncing d2: Mirror State: Okay State: Resyncing Resync in progress: 7 % done d0: Submirror of d2 State: Okay d1: Submirror of d2 State: ResyncingIn effort to be thorough, we install the boot block to the mirror disk
and configure grub for the alternate boot path:
snorkle [0] /usr/sbin/installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c1t1d0s0 stage1 written to partition 0 sector 0 (abs 16065) stage2 written to partition 0, 272 sectors starting at 50 (abs 16115) snorkle [0] /usr/bin/cat << EOF >> /boot/grub/menu.lst > title Solaris 10 10/09 root mirror > root (hd1,0,a) > kernel /platform/i86pc/multiboot -mverbose > module /platform/i86pc/boot_archive > EOF snorkle [0] /usr/sbin/bootadm list-menu The location for the active GRUB menu is: /boot/grub/menu.lst default 0 timeout 10 0 Solaris 10 10/09 s10x_u8wos_08a X86 1 Solaris failsafe 2 Solaris 10 10/09 root mirrorAlternatively, for sparc:
sparc [0] /usr/sbin/installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c1t1d0s0
sparc [0] /usr/sbin/eeprom use-nvramrc\?=true
sparc [0] /usr/bin/ls -ld /dev/rdsk/c1t0d0s0 /dev/rdsk/c1t1d0s0 | /usr/bin/awk '{print $9, $11}'
/dev/rdsk/c1t0d0s0 ../../devices/pci@1c,600000/scsi@2/sd@0,0:a,raw
/dev/rdsk/c1t1d0s0 ../../devices/pci@1c,600000/scsi@2/sd@1,0:a,raw
sparc [0] /usr/sbin/eeprom "nvramrc=devalias rootdisk /pci@1c,600000/scsi@2/sd@0,0:a devalias rootmirror /pci@1c,600000/scsi@2/sd@1,0:a"
sparc [0] /usr/sbin/eeprom "boot-device=rootdisk rootmirror"
Lastly, setup the dump device for crash dumps:snorkle [0] /usr/sbin/dumpadm -d /dev/md/dsk/d5 Dump content: kernel pages Dump device: /dev/md/dsk/d5 (swap) Savecore directory: /var/crash/snorkle Savecore enabled: yes
COMMANDS (quick)
snorkle [0] /usr/sbin/reboot -- 'cdrom -s' sol10cdrom [0] /usr/sbin/fsck -y /dev/rdsk/c1t0d0s0 sol10cdrom [0] [ ! -d /a ] && /usr/bin/mkdir /a sol10cdrom [0] /usr/sbin/mount /dev/dsk/c1t0d0s0 /a sol10cdrom [0] /usr/bin/cp /a/etc/system /a/etc/system.orig sol10cdrom [0] /usr/bin/cp /a/etc/vfstab /a/etc/vfstab.orig sol10cdrom [0] /usr/bin/sed -e 's;rootdev:/pseudo/vxio@0:0;*rootdev:/pseudo/vxio@0:0;' /a/etc/system.orig > /a/etc/system sol10cdrom [0] /usr/bin/cp /a/etc/system /a/etc/system.orig1 sol10cdrom [0] /usr/bin/sed -e 's;set vxio:vol_rootdev_is_volume=1;*set vxio:vol_rootdev_is_volume=1;' /a/etc/system.orig1 > /a/etc/system# update the current 'vfstab' to this:
sol10cdrom [0] /usr/bin/cat /a/etc/vfstab #device device mount FS fsck mount mount #to mount to fsck point type pass at boot options # fd - /dev/fd fd - no - /proc - /proc proc - no - #/dev/vx/dsk/bootdg/swapvol - - swap - no - #/dev/vx/dsk/bootdg/rootvol /dev/vx/rdsk/bootdg/rootvol / ufs 1 no - #/dev/vx/dsk/bootdg/var /dev/vx/rdsk/bootdg/var /var ufs 1 no - /dev/dsk/c1t0d0s1 - - swap - no - /dev/dsk/c1t0d0s0 /dev/rdsk/c1t0d0s0 / ufs 1 no - /dev/dsk/c1t0d0s3 /dev/rdsk/c1t0d0s3 /var ufs 1 no - /devices - /devices devfs - no - sharefs - /etc/dfs/sharetab sharefs - no - ctfs - /system/contract ctfs - no - objfs - /system/object objfs - no - swap - /tmp tmpfs - yes - #vshare - /mnt/vshare vboxfs - no - #/dev/vx/dsk/mydg/myvol /dev/vx/rdsk/mydg/myvol /mnt/myvol vxfs 1 no - #NOTE: volume rootvol (/) encapsulated partition c1t0d0s0 #NOTE: volume swapvol (swap) encapsulated partition c1t0d0s1 #NOTE: volume var (/var) encapsulated partition c1t0d0s3
sol10cdrom [0] /usr/bin/touch /a/etc/vx/reconfig.d/state.d/install-db sol10cdrom [0] /usr/bin/rm /a/etc/vx/reconfig.d/state.d/root-done sol10cdrom [0] /usr/sbin/bootadm update-archive -R /a sol10cdrom [0] /usr/sbin/reboot -- -s snorkle [0] /usr/sbin/df -h snorkle [0] /usr/bin/rm /etc/vx/reconfig.d/state.d/install-db snorkle [0] /usr/sbin/vxiod set 10 snorkle [0] /usr/sbin/vxconfigd -m disable snorkle [0] /usr/sbin/vxdctl enable snorkle [0] /usr/sbin/vxedit -g rootdg -rf rm rootvol snorkle [0] /usr/sbin/vxedit -g rootdg -rf rm swapvol snorkle [0] /usr/sbin/vxedit -g rootdg -rf rm var snorkle [0] /usr/sbin/vxdg -g rootdg rmdisk rootmirror snorkle [0] /usr/sbin/vxdg destroy rootdg snorkle [0] /usr/sbin/vxvol -g mydg startall snorkle [0] /usr/sbin/mount /mnt/myvol snorkle [0] /etc/vx/bin/vxedvtoc -f /etc/vx/reconfig.d/disk.d/c1t0d0/vtoc /dev/rdsk/c1t0d0s2 DO YOU WANT TO WRITE THIS TO THE DISK ? [Y/N] :y WRITING THE NEW VTOC TO THE DISK snorkle [0] echo "set md:mirrored_root_flag=1" >> /etc/system snorkle [0] /usr/bin/mv /boot/grub/menu.lst /boot/grub/menu.lst.vxvm snorkle [0] /usr/bin/mv /boot/grub/menu.lst.pre_vxvm /boot/grub/menu.lst snorkle [0] /usr/sbin/bootadm update-archive -v snorkle [0] /usr/bin/touch /etc/vx/reconfig.d/state.d/install-db snorkle [0] /usr/sbin/reboot -- 'cdrom -s'
# output of 'format c1t0d0' -> verify pre modification Part Tag Flag Cylinders Size Blocks 0 root wm 1 - 1045 8.01GB (1045/0/0) 16787925 1 swap wu 1046 - 1189 1.10GB (144/0/0) 2313360 2 backup wm 0 - 1333 10.22GB (1334/0/0) 21430710 3 var wm 1190 - 1333 1.10GB (144/0/0) 2313360 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0 8 boot wu 0 - 0 7.84MB (1/0/0) 16065 9 unassigned wu 0 0 (0/0/0) 0# output of 'format c1t0d0' -> verify post modification
Part Tag Flag Cylinders Size Blocks 0 root wm 1 - 1045 8.01GB (1045/0/0) 16787925 1 swap wu 1050 - 1189 1.07GB (140/0/0) 2249100 2 backup wm 0 - 1333 10.22GB (1334/0/0) 21430710 3 var wm 1190 - 1333 1.10GB (144/0/0) 2313360 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 1046 - 1047 15.69MB (2/0/0) 32130 6 unassigned wm 1048 - 1049 15.69MB (2/0/0) 32130 7 unassigned wm 0 0 (0/0/0) 0 8 boot wu 0 - 0 7.84MB (1/0/0) 16065 9 unassigned wu 0 0 (0/0/0) 0
sol10cdrom [0] /usr/sbin/fsck -n /dev/rdsk/c1t0d0s0 sol10cdrom [0] /usr/sbin/reboot -- -s snorkle [0] /usr/sbin/prtvtoc -h /dev/rdsk/c1t0d0s2 | /usr/sbin/fmthard -s - /dev/rdsk/c1t1d0s2 snorkle [0] /usr/sbin/metadb -a -c3 -f c1t0d0s5 c1t0d0s6 c1t1d0s5 c1t1d0s6 snorkle [0] /usr/sbin/metainit -f d0 1 1 c1t0d0s0 snorkle [0] /usr/sbin/metainit d2 -m d0 snorkle [0] /usr/sbin/metainit -f d3 1 1 c1t0d0s1 snorkle [0] /usr/sbin/metainit d5 -m d3 snorkle [0] /usr/sbin/metainit -f d6 1 1 c1t0d0s3 snorkle [0] /usr/sbin/metainit d8 -m d6 snorkle [0] /usr/sbin/metaroot d2# update vfstab to this
snorkle [0] /usr/bin/cat /etc/vfstab #device device mount FS fsck mount mount #to mount to fsck point type pass at boot options # fd - /dev/fd fd - no - /proc - /proc proc - no - /dev/md/dsk/d5 - - swap - no - /dev/md/dsk/d2 /dev/md/rdsk/d2 / ufs 1 no - /dev/md/dsk/d8 /dev/md/rdsk/d8 /var ufs 1 no - /devices - /devices devfs - no - sharefs - /etc/dfs/sharetab sharefs - no - ctfs - /system/contract ctfs - no - objfs - /system/object objfs - no - swap - /tmp tmpfs - yes - #vshare - /mnt/vshare vboxfs - no - /dev/vx/dsk/mydg/myvol /dev/vx/rdsk/mydg/myvol /mnt/myvol vxfs 1 no - #NOTE: volume d2 (/) svm c1t[0|1]d0s0 #NOTE: volume d5 (swap) svm c1t[0|1]d0s1 #NOTE: volume d8 (/var) svm c1t[0|1]d0s3
snorkle [0] /usr/bin/rm /etc/vx/reconfig.d/state.d/install-db snorkle [0] /usr/sbin/bootadm update-archive -v snorkle [0] /usr/sbin/reboot snorkle [0] /usr/sbin/metainit d1 1 1 c1t1d0s0 snorkle [0] /usr/sbin/metattach d2 d1 snorkle [0] /usr/sbin/metainit d4 1 1 c1t1d0s1 snorkle [0] /usr/sbin/metattach d5 d4 snorkle [0] /usr/sbin/metainit d7 1 1 c1t1d0s3 snorkle [0] /usr/sbin/metattach d8 d7 snorkle [0] /usr/sbin/metastat | /usr/bin/egrep '^d|State:|in progress' snorkle [0] /usr/sbin/installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c1t1d0s0 snorkle [0] /usr/bin/cat << EOF >> /boot/grub/menu.lst > title Solaris 10 10/09 root mirror > root (hd1,0,a) > kernel /platform/i86pc/multiboot -mverbose > module /platform/i86pc/boot_archive > EOF snorkle [0] /usr/sbin/bootadm list-menu# for sparc hosts:
sparc [0] /usr/sbin/installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c1t1d0s0
sparc [0] /usr/sbin/eeprom use-nvramrc\?=true
sparc [0] /usr/bin/ls -ld /dev/rdsk/c1t0d0s0 /dev/rdsk/c1t1d0s0 | /usr/bin/awk '{print $9, $11}'
/dev/rdsk/c1t0d0s0 ../../devices/pci@1c,600000/scsi@2/sd@0,0:a,raw
/dev/rdsk/c1t1d0s0 ../../devices/pci@1c,600000/scsi@2/sd@1,0:a,raw
sparc [0] /usr/sbin/eeprom "nvramrc=devalias rootdisk /pci@1c,600000/scsi@2/sd@0,0:a devalias rootmirror /pci@1c,600000/scsi@2/sd@1,0:a"
sparc [0] /usr/sbin/eeprom "boot-device=rootdisk rootmirror"
# don't forget the dump device:snorkle [0] /usr/sbin/dumpadm -d /dev/md/dsk/d5
NOTES
1) The above procedure will NOT work if VxVM is less than version 4.0
and still needs to maintain other volumes. This is due to a VxVM
requirement (in prior versions) on the existence of a 'rootdg'.
2) It is assumed that the root disk is still encapsulated while under
VxVM control, having not been moved to an initialized disk. The above
will only work with an encapsulated disk.
3) The above will definitely incur downtime as the majority of the work
is performed from single user as well as booted single user from
the cdrom. The amount of downtime will vary depending upon a
variety of factors such as size of the volumes, hardware platform
and capabilities, etc.
4) This step requires deliberate modification of the partition table
of the root disk, which is also the reason that it requires booting
from an alternate device such as the cdrom. Care must be taken in
this step. Detailed is the adjustment of 'swap' space to create two
new slices. This works because there are no FS constructs setup for
'swap'. To modify any other existing slice would most likely result
in data loss and FS corruption. Of all the steps presented this one is
probably the most likely to cause problems if not performed correctly.
A good understanding of disk partitions and structures would be useful.
As an aside, you cannot realistically perform this steps if there are
no free slices available. This step also assumes all free space on the
disk has already been allocated. If free space is available, use it
rather than mucking with 'swap'.
