A short guide to the Solaris Loop-back file systems and mounting ISO images

The Solaris Loop-back file system is a handy bit of software, allowing you to "mount" directories, files and, in particular, CD or DVD image files in ISO-9660 format.

To make it more user friendly, build 91 of ONV introduces the ability to the mount command to automatically create the loop-back devices for ISO images! The Changelog for NV 91 has got the following note:

Issues Resolved: PSARC case 2008/290 : lofi mount BUG/RFE:6384817Need persistent lofi based mounts and direct mount(1m) support for lofi

In older releases, it was necessary to run two commands to mount an ISO image file. The first to set up a virtual device for the ISO image:

# lofiadm -a /shared/Downloads/image.iso /dev/lofi/1

And then to mount it somewhere:

# mount -F hsfs -o ro /dev/lofi/1 /mnt

Solaris uses hsfs to indicate the "High Sierra File System" driver used to mount ISO-9660 files. Specify "-o ro" to make it Read-only, though that is the default for hsfs file systems, at least lately (I seem to recall that at one point in the past it was mandatory to specify read-only mounting explicitly.

Looking at what has been happening here, we can see the Loop-back device by running lofiadm without any options:

# lofiadm Block Device File Options /dev/lofi/1 /shared/Downloads/image.iso -

And the mounted file system:

# df -k /mnt Filesystem kbytes used avail capacity Mounted on /dev/lofi/1 2915052 2915052 0 100% /mnt

The new feature of the mount command requires a full path to the ISO file (Just like lofiadm does, at any rate it does for now)

# mount -F hsfs -o ro /shared/Downloads/image2.iso /mnt

To check the status:

# df -k /mnt Filesystem kbytes used avail capacity Mounted on /shared/Downloads/image2.iso 7781882 7781882 0 100% /mnt

And when we run lofiadm we see it automatically created a new device, /dev/lofi/2:

# lofiadm Block Device File Options /dev/lofi/1 /shared/Downloads/image.iso - /dev/lofi/2 /shared/Downloads/image2.iso -

Some of the other uses of the Loop-back file system:

You can mount any directory on any other directory:

# mkdir /mnt1 # mount -F lofs -o ro /usr/spool/print /mnt2

Note the use of lofs as the file system "type". This is a bit like a hard-link to a directory, and it can exist across file systems. These can be read-write or read-only.

You can also mount any individual file onto another file:

# mkdir /tmp/mnt # echo foobar > /tmp/mnt/X # mount -F lofs /usr/bin/ls /tmp/mnt/X # ls -l /tmp/mnt total 67 -r-xr-xr-x 1 root bin 33396 Jun 16 05:43 X # cd /tmp/mnt # ./X X # ./X -l total 67 -r-xr-xr-x 1 root bin 33396 Jun 16 05:43 X

The above feature incidentally inspired item nr 10 on my ZFS feature wish list.

This allows for a lot of flexibility. In deed this functionality is central to how file systems and disk space is provisioned in Solaris Zones. If you play around with it you will find plenty of uses for it!

Using a dedicated ZFS file system to manage process core dumps

ZFS just bristles with potential. Quotas, Reservations, turning compression or atime updates on or off without unmounting. The list goes on.

So now that we have ZFS root (Since Nevada build SNV_90, and even earlier when using OpenSolaris or other distributions) lets start to make use of these features.

First thing is, on my computer I don't care about access time updates on files or directories, so I disable it.

/ $ pfexec zfs set atime=off rpool

That is not particularly spectacular in itself, but since it is there I use it. The idea is of course to save a few disk updates and the corresponding IOs.

Next: core dumps. One of my pet hates. Many processes dumps core in your home dir, these get overwritten or forgotten, and then there are any number of core files lying around all over the file systems, all off these just wasting space since I don't really intent do try to analyze any of them.

Solaris has got a great feature by which core dumps can be all directed to go to a single directory and, on top of that, to have more meaningful file names.

So the idea is to create a directory, say /var/cores and then store the core files in there for later review. But knowing myself these files will just continue to waste space until I one day decide to actually try and troubleshoot a specific issue.

To me this sounds like a perfect job for ZFS.

First I check that there is not already something called /var/cores:

/ $ ls /var/cores /var/cores: No such file or directory

Great. Now I create it.

/ $ pfexec zfs create rpool/export/cores / $ pfexec zfs set mountpoint=/var/cores rpool/export/cores

And set a limit on how much space it can ever consume:

/ $ pfexec zfs set quota=2g rpool/export/cores

Note: This can easily be changed at any time, simply by setting a new quota.

Which creates the below picture.

/ $ df -h Filesystem size used avail capacity Mounted on rpool/ROOT/snv_91 20G 5.9G 7.0G 46% / /devices 0K 0K 0K 0% /devices /dev 0K 0K 0K 0% /dev ctfs 0K 0K 0K 0% /system/contract proc 0K 0K 0K 0% /proc mnttab 0K 0K 0K 0% /etc/mnttab swap 2.3G 416K 2.3G 1% /etc/svc/volatile objfs 0K 0K 0K 0% /system/object sharefs 0K 0K 0K 0% /etc/dfs/sharetab /usr/lib/libc/libc_hwcap1.so.1 13G 5.9G 7.0G 46% /lib/libc.so.1 fd 0K 0K 0K 0% /dev/fd swap 2.3G 7.2M 2.3G 1% /tmp swap 2.3G 64K 2.3G 1% /var/run rpool/export 20G 19K 7.0G 1% /export rpool/export/home 20G 6.8M 7.0G 1% /export/home rpool 20G 44K 7.0G 1% /rpool rpool/export/cores 2.0G 18K 2.0G 1% /var/cores SHARED 61G 24K 31G 1% /shared ... snip ...

And checking the settings on the /var/cores ZFS file system

/ $ zfs get all rpool/export/cores NAME PROPERTY VALUE SOURCE rpool/export/cores type filesystem - rpool/export/cores creation Thu Jun 19 14:18 2008 - rpool/export/cores used 18K - rpool/export/cores available 2.00G - rpool/export/cores referenced 18K - rpool/export/cores compressratio 1.00x - rpool/export/cores mounted yes - rpool/export/cores quota 2G local rpool/export/cores reservation none default rpool/export/cores recordsize 128K default rpool/export/cores mountpoint /var/cores local rpool/export/cores sharenfs off default rpool/export/cores checksum on default rpool/export/cores compression off default rpool/export/cores atime off inherited from rpool rpool/export/cores devices on default rpool/export/cores exec on default rpool/export/cores setuid on default rpool/export/cores readonly off default rpool/export/cores zoned off default rpool/export/cores snapdir hidden default rpool/export/cores aclmode groupmask default rpool/export/cores aclinherit restricted default rpool/export/cores canmount on default rpool/export/cores shareiscsi off default rpool/export/cores xattr on default rpool/export/cores copies 1 default rpool/export/cores version 3 - rpool/export/cores utf8only off - rpool/export/cores normalization none - rpool/export/cores casesensitivity sensitive - rpool/export/cores vscan off default rpool/export/cores nbmand off default rpool/export/cores sharesmb off default rpool/export/cores refquota none default rpool/export/cores refreservation none default

Note that Access-time updates on this file system is off - the setting has been inherited from the pool. The only "local" settings are the mountpoint and the quota which corresponds to the items that I've specified manually.

Now just to make new core files actually use this directory. At present, the default settings from coreadm looks like this:

/ $ coreadm global core file pattern: global core file content: default init core file pattern: core init core file content: default global core dumps: disabled per-process core dumps: enabled global setid core dumps: disabled per-process setid core dumps: disabled global core dump logging: disabled

Looking at the coreadm man page, there is a fair amount of flexibility in what can be done. I want core files to have a name identifying the zone in which the process was running, the process executable file, and the user. I also don't want core dumps to overwrite when the same process keeps on faulting, so I will add a time stamp to the core file name.

/ $ pfexec coreadm -g /var/core/core.%z.%f.%u.%t

And then I would like to enable logging of an event any time when a core file is generated, and also to store core files for Set-UID processes:

/ $ pfexec coreadm -e global-setid -e log

And finally, just to review the core-dump settings, these now look like this:

/ $ coreadm global core file pattern: /var/core/core.%z.%f.%u.%t global core file content: default init core file pattern: core init core file content: default global core dumps: disabled per-process core dumps: enabled global setid core dumps: enabled per-process setid core dumps: disabled global core dump logging: enabled

Now if that is not useful, I don't know what is! You will soon start to appreciate just how much space is wasted and just how truly rigid and inflexible other file systems are once you run your machine with a ZFS root!

Update: More on how to make x86 Solaris with Grub boot verbosely

Since I posted a while ago on how to make Solaris boot verbosely, I found a better way. Or rather, I learned a bit more about this.

In stead of just adding -v to the kernel line, add "-v -m verbose"

The "-m verbose" portion passes the verbose option to SMF, giving you verbose information about startup of services.

The "-v" causes the messages which normally goes to the system log to also be emitted on the console.

My grub entry for verbose booting now looks like this:

# Solaris SNV91 Verbose Boot title Solaris SNV_91 Verbose Boot findroot (BE_SNV_91,1,a) kernel$ /platform/i86pc/kernel/$ISADIR/unix -v -m verbose -B $ZFS-BOOTFS module$ /platform/i86pc/$ISADIR/boot_archive # End Solaris SNV_91 Verbose

Of course you don't need the entry to be in grub - if your system is not booting, use the edit feature to add these options to the kernel line in the grub item you want to boot from.

Also just a note on the spash image. Removing it is entirely optional - not removing it will not hide any bootup messages (as I previously through)

Not a day goes by that I don't learn something new about Solaris.

Sharing a ZFS pool between Linux and Solaris

If you are multi-booting between Linux and Solaris (and others like FreeBSD, OpenBSD and Mac OS X, I expect) you will sooner or later encounter the problem of how to share disk space between the operating systems. FAT32 is not satisfactory due to its lack of POSIX features, in particular file ownership and access modes, not to mention its sub par performance. ext2/3 is not an option because you only get read-only support for it in Solaris, and similarly UFS enjoys only read-only support in Linux. The whole situation is rather depressing.

Enter ZFS.

This all started because I discovered that I can have a ZFS root file system without having to install OpenSolaris. The trick as some of you may know, is to select "Solaris Express" from the first menu on booting the install disk, and then select one of the two "Interactive Text" options from the next menu. This puts you back into 1984 in terms of installers, but you get the option of using ZFS for root!

Note: It might be possible to do this with the default installer, but on my computer the installer just would not run (I got some daft error about fonts and mouse themes). With a ZFS root, the Swap and Dump automatically goes onto dedicated vdevs, and you save a lot in terms of pre-allocated space.

I have of course used ZFS on my laptop previously as a test, but the benefits were limited by the fact that I still had "slices" for the OS and a small ZFS pool on a spare slice.

I'm not sure which build of Nevada first introduced the ZFS root option in the installer, but it is available in build 90 at least.

My choice of Linux distribution is Ubuntu 8.04. The steps to setting up a ZFS pool shared across operating systems are as follow:

1. Select a Partitioning scheme with minimal space allocated to each of Ubuntu and Nevada.
I decided to put Ubuntu in an Extended partition with a 10 GB Logical Partition for the OS, /var and /home, and a 1 GB Logical partition for Swap.
For Solaris I allocated a 24 GB primary partition to become the ZFS root pool, which includes Swap, Dump, OS and Live-upgrade space.
The balance of the 100 GB disk will be shared between Ubuntu and Solaris using ZFS.

Note: Linux and Solaris has got some different views on how disk partitioning works.
Due to historical reasons, in particular due to compatibility with Solaris on SPARC hardware, Solaris slices live in a single primary partition with an identifier of 0x82 (SOLARIS) or 0xbf (SOLARIS2) somewhat like how logical fdisk partitions live inside an "extended partition".

2. Install Ubuntu first, creating only the partitions for it. Remember to not have any external drives connected as it can screw up the order in which drives are detected and as a result bugger up the Grub menu list.

During the installation you create an Admin user. This will eventually in the future become a "backup" admin user.

3. Reboot and load patches/updates, and backup the Grub /boot/grub/menu.lst file to an external media such as a USB thumb drive for easy access. The Ubuntu Grub does not understand ZFS, so you need to use Nevada's Grub to manage the multi-booting.

4. Also set Ubuntu to use the hardware clock as local time in stead of UTC. (This is what Solaris uses) To do this change UTC=yes to UTC=no in /etc/default/rcS, then reboot.

5. Install Nevada. Use either of the Interactive Text installer options, but for simplicity's sake specify the system as non-networked.

6. Reboot and create a user for every-day use, and add this user to the "Primary Administrator" role using usermod -P "Primary Administrator" <userid>

7. Add the Ubuntu Grub entries you saved in step 3 to the end of the Nevada grub menu.lst file. This will be stored in /boot/grub/menu.lst (The default pool name is rpool)

8. Reboot back into Ubuntu, then follow the Linux ZFS-FUSE instalation instructions to get ZFS-FUSE installed. I used the trunk to get the latest ZFS updates from Opensolaris.org included. Also see this Ubuntu Wiki page, and Ralf Hildebrand's blog for more info.

For reference, this is the procedure I used

apt-get install mercurial build-essential scons libfuse-dev libaio-dev devscripts build-essential zlib1g-dev cd ~ hg clone http://www.wizy.org/mercurial/zfs-fuse/trunk cd trunk/src scons sudo scons install

9. Create an fdisk partition for the shared ZFS pool using the remaining disk space. I used a primary partition and set the identifier to W95 FAT32, though this is probably unimportant.

10. While still running running Ubuntu, create a ZFS pool on this new fdisk partition using a command like this:

sudo /usr/local/sbin/zfs-fuse sudo /usr/local/sbin/zpool create -m /shared SHARED

I like to give my ZFS pools names in all-capitals, purely because it makes the ZFS pool devices stand out better in the output from df and mount.

WARNING: I found that if I created the ZFS pool under Solaris, it refused to import into Ubuntu, but if I created it under Linux it imports/exports just fine in both directions. Both pools are created as version 10 pools, so the reason for this is not obvious. If you do decide to experiment with creating the pool under Solaris, when you want to realy get rid of the pool you will discover you need to dd zeros over the pool before creating it again, otherwise the condition remains unchanged despite destroying and re-creating the pool. If you do experiment with this please do share your results!

11. Export the ZFS pool using

/usr/local/sbin/zpool export SHARED

12. Reboot into Nevada and import the pool using

/usr/local/sbin/zpool import SHARED

Note: If you forget to export before you shut down, you will need to add -f to force the import after booting into the other OS.

At this point I just sat there and stared in wonder at how well it actually works. There is beauty in finally seeing this working!

13. Create some init.d / rc scripts to automate the import/export on shutdown/startup.

14. Now you can start customizing both operating environments. You may want to setup Automatic network configuration by enabling the SMF for NWAM in Solaris, eg by doing:

pfexec svcadm disable physical:default pfexec svcadm enable physical:nwam

I'm looking forward to testing Live Upgrade on my setup with ZFS root, and to getting a shared home directory to work well for both Solaris and Ubuntu. I have created a login ID with the same gid/uid and a home directory under the shared ZFS pool, but after a few changes it got broken under Ubuntu, probably due to subtle differences in how Gnome/Desktop config items are stored and/or expected.

Despite my initial sceptism about FUSE, it is actually quite functional. All-in-all I love being able to share a file system, well, many files systems, between the two operating environments!

How to get Solaris with Grub on x86 to boot verbosely

The fact is, great as OpenSolaris is, things do sometimes go wrong, and often booting with verbose output will help you identify why, or at least where in the boot process, things are breaking. How to enable verbose booting is a really simple question, yet one to which I found the answer to be quite elusive for months. I still don't have an official answer, but this is how I managed to do it:

First, find your grub boot menu list file. If you are using ZFS boot, it is in /rpool/boot/grub and if you are using UFS it will be in /boot/grub

Make a backup of the file, then open it up in your favorite editor, and comment out the splash image line near the start.

Now copy the section for your Solaris image. On my system it looks like this:

title LiveUpgradeSNV90_2008-05-27 findroot (BE_LiveUpgradeSNV90_2008-05-27,0,e) kernel$ /platform/i86pc/kernel/$ISADIR/unix module$ /platform/i86pc/$ISADIR/boot_archive

You need to change two things: 1) Give it a new title, and 2) Add -v to the kernel line.

Note: -v is in fact a boot-arg, supplied to the unix program. For detailed information have a look at the man page for boot(1m)

After saving your changes, use bootadm to check for mistakes, eg

# bootadm list-menu The location for the active GRUB menu is: /boot/grub/menu.lst default 3 timeout 10 0 LiveUpgradeSNV89_2008-05-21 1 LiveUpgradeSNV89_2008-05-21 Solaris xVM 2 LiveUpgradeSNV89_2008-05-21 failsafe 3 LiveUpgradeSNV90_2008-05-27 4 LiveUpgradeSNV90_2008-05-27 Solaris xVM 5 LiveUpgradeSNV90_2008-05-27 failsafe 6 Verbose_Boot_LiveUpgradeSNV90_2008-05-27

Entry nr 6 is the new Entry I added by making a copy of entry nr 3 in the list.

Now all that remains is to test it by rebooting the box. Q.E.D.

Automating Solaris Zone creation

Zones can be treated as cheap, disposable application containers. Automated Zone creation is not necessarily there to allow you to rapidly deploy 1000s of Zones (though it could certainly be used for this purpose given sufficient planning), but allows you to create and use, then delete and easily re-create zones freshly and with a consistent configuration.

You will find that most, if not all, of your zones will use the same naming-services configuration, be in the same time-zone, attach to the same network interface (just with different IP addresses), etc. Many of the System Identification and system configuration settings will be identical or very similar between the Zones.

You might even find that with each new zone you create the same set of user-ids and have them all get their home directories from a central home-directory server. Basically repeat work. Computers are, in fact, good at repeatedly doing the same task over and over, without getting bored.

If all you want to achieve is to have a clean state to which you can restore a zone easily, then a fine plan would be to use file system snapshots, something like this:

1. Preparation / Setup

1.1. Create a file system structure in which to store the Zone. Since we've got ZFS for free with Solaris there's really no reason not to use it.

1.2. Set up the Zone in this file system, and complete the configuration up to the point where you want to be able to revert back to.

1.3. Shut down the Zone and take a snapshot.

2. Using this Zone:

2.1 Make any instance specific "custom" configuration changes (add some disk space, user-ids, tweak some settings)

2.2 Start the zone and let the users loose in it.

3. Reverting to the clean status

3.1 Bring the zone down (purely to make sure that no processes have files open in the file system containing the zone)

3.2 Recover the file system back to the Snap-shot state.

3.3 Go back to nr 2 above.

Before I show an example of doing this using ZFS, suffer me to mention the other techniques involved in automating Solaris Zone creation (Each of which I will cover in a separate blog post in detail)

Firstly copying the Zone configuration. This involves creating a zone config and exporting it to a file to be used as a template in the future. Then each time you want to create a zone based on this template, you just make a few small changes such as the zone-name and IP address, then import this modified copy of the template into a new zone, after which you continue with the normal zone installation.

Using a sysidcfg file and a few other tricks to speed up the zone configuration is quite similar to using a sysidcfg file to pre-configure a system from a jumpstart, and can by used to automate settings such as the timezone, locale, terminal type, networking, and name-services, amongst others.

Cloning Zones to speed up the install process. The Zone management framework from Sun gives us the ability to "clone" a master "template" zone. This involves creating one (or more) template zones which you then leave fully installed and configured, but don't actually ever start up or use, other than to tweak their configurations. This saves time during the actual install and subsequent configuration steps.

With that out of the way, on to the example of how to make a simple disposable Zone. As always the fixed-width text represents what you should see on the screen. I highlight the bits you enter.

globalzone# zpool create SPACE c0d0s4

globalzone# zfs create SPACE/zones

globalzone# zfs set mountpoint=/export/zones SPACE/zones

globalzone# zfs create SPACE/zones/disposable

globalzone# chmod 0700 /export/zones/disposable

globalzone# zfs set atime=off SPACE/zones/disposable

Disabling of “atime” above is a personal preference thing. Now we set up a simple zone. Yours can be as complicated or as simple as you want it to be.

globalzone# zonecfg -z disposable

zonecfg:disposable> set zonepath=/export/zones/disposable

zonecfg:disposable> add net

zonecfg:disposable:net> set physical=e1000g0

zonecfg:disposable:net> set address=192.168.24.133

zonecfg:disposable:net> end

zonecfg:disposable> verify

zonecfg:disposable> commit

zonecfg:disposable> exit

globalzone# zoneadm -z disposable install

cannot create ZFS dataset SPACE/zones/disposable: dataset already exists

Preparing to install zone .

Creating list of files to copy from the global zone.

Copying <9386> files to the zone.

Initializing zone product registry.

Determining zone package initialization order.

Preparing to initialize <1307> packages on the zone.

Initialized <1307> packages on zone.

Zone is initialized.

Installation of <1> packages was skipped.

Installation of these packages generated warnings:

The file contains a log of the zone installation.

For the eagle-eyed amongst you, the WebStackTooling failure is due to the fact that this is a sparse zone and I'm running beta software (Nevada Build 80). In a sparse zone the /usr file system is read-only and The WebStackTooling is trying to create or change some files. I'm just ignoring this error for now as it does not bother me.

So far, so good. Lets save a backup of what we've got so far.

globalzone# zfs snapshot SPACE/zones/disposable@freshly_installed

Now we perform the first boot and system identification. Below is an abbreviated copy-paste showing the flow of the process.

globalzone# zoneadm -z disposable boot; zlogin -C disposable

[Connected to zone 'disposable' console]

Configuring Services ... 150/150

Reading ZFS config: done.

>>> Select a Language

>>> Select a Locale

>>> What type of terminal are you using?

Creating new rsa public/private host key pair

Creating new dsa public/private host key pair

Configuring network interface addresses: e1000g0.

>>> Host name for e1000g0:1 disposable

>>> Configure Security Policy:

>>> Name Service

>>> NFSv4 Domain Name:

>>> Region and Time zone: Africa/Johannesburg

>>> Root Password

System identification is completed.

rebooting system due to change(s) in /etc/default/init

[NOTICE: Zone rebooting]

SunOS Release 5.11 Version snv_80 64-bit

Copyright 1983-2007 Sun Microsystems, Inc. All rights reserved.

Use is subject to license terms.

Hostname: disposable

Reading ZFS config: done.

disposable console login: root

Password:

Feb 20 21:33:05 disposable login: ROOT LOGIN /dev/console

Sun Microsystems Inc. SunOS 5.11 snv_80 January 2008

You may want to make a few more changes now that the Zone is running. Some ideas may be to set up User-IDs, enable/disable some services, and set up some NFS and/or autmounter file systems.

# mkdir /export/home

# useradd -c "Joe Blogs" -d /export/home/joeblogs -m joeblogs

# passwd joeblogs

Assuming you've done all you want, this is the point where we have a cleanly built zone, running, and essentially the point that we would like to be able to return to after we did whatever make-and-break or sandbox testing. The Zone should be halted before we take the snapshot, even if only to close all open files.

# halt

Feb 20 21:33:12 disposable halt: initiated by root on /dev/console

Feb 20 21:33:12 disposable syslogd: going down on signal 15

[NOTICE: Zone halted]

~.

[Connection to zone 'disposable' console closed]

Now just take another ZFS snapshot:

globalzone# zfs snapshot SPACE/zones/disposable@system_identified

=================

Now the Zone is ready for you to let your users loose in it. Allow them to have full root access, go crazy, run "rm -r /", etc.

globalzone# zoneadm -z disposable boot; zlogin -C disposable

zoneadm: zone 'disposable': WARNING: e1000g0:1: no matching subnet found in netmasks(4) for 192.168.24.133; using default of 255.255.255.0.

[Connected to zone 'disposable' console]

Hostname: disposable

Reading ZFS config: done.

disposable console login: root

Password:

Feb 20 21:40:11 disposable login: ROOT LOGIN /dev/console

Last login: Wed Feb 20 21:33:05 on console

Sun Microsystems Inc. SunOS 5.11 snv_80 January 2008

Now perform some "work" - Create a few directories, modify some files, etc. I chose to run sys-unconfig.

# sys-unconfig

WARNING

This program will unconfigure your system. It will cause it

to revert to a "blank" system - it will not have a name or know

about other systems or networks.

This program will also halt the system.

Do you want to continue (y/n) ? y

sys-unconfig started Wed Feb 20 21:40:30 2008

sys-unconfig completed Wed Feb 20 21:40:30 2008

Halting system...

svc.startd: The system is coming down. Please wait.

svc.startd: 59 system services are now being stopped.

svc.startd: The system is down.

[NOTICE: Zone halted]

Then, back in the global zone, examine the available ZFS snapshots:

globalzone# zfs list

NAME USED AVAIL REFER MOUNTPOINT

SPACE 684M 14.1G 18K /SPACE

SPACE/zones 684M 14.1G 19K /export/zones

SPACE/zones/disposable 684M 14.1G 624M /export/zones/disposable

SPACE/zones/disposable@freshly_installed 790K - 523M -

SPACE/zones/disposable@system_identified 59.2M - 611M -

These four commands can go nicely into a little "revert" script.

globalzone# zfs clone SPACE/zones/disposable@system_identified \

SPACE/zones/reverted_temp

globalzone# zfs promote SPACE/zones/reverted_temp

globalzone# zfs destroy SPACE/zones/disposable

globalzone# zfs rename SPACE/zones/reverted_temp SPACE/zones/disposable

That took just a few seconds, and we are ready to start using the zone again...

global# zoneadm -z disposable boot; zlogin -C disposable

[Connected to zone 'disposable' console]

SunOS Release 5.11 Version snv_80 64-bit

Copyright 1983-2007 Sun Microsystems, Inc. All rights reserved.

Use is subject to license terms.

Hostname: disposable

Reading ZFS config: done.

disposable console login:

As expected, you will find that all changes are reverted. Besides the normal application test environment, one other area where I think this would be quite handy is in a class-room situation, where you can allow the students full root access in the zone, and at the end of the day quickly recover the system to a sane state for the next day's class.

All in all that was Q-E-D. This principle, as well as the information from my previous blog posting will form the basis of the next few posts.

15 minutes to your first Solaris zone

So you got OpenSolaris on your local machine and you would like to try out Zones. This tutorial will give you a 15 minute by-example guide to setting up your first zone, and after that explain a few things hands-on, showing you where what is.

There are a few things to understand about Zones before we start:

First thing is the concept of the Global Zone - the "base" operating system instance is the Global zone and all other "local zones" run under the control of the global zone.

Secondly you get three types of local Zones, being Full root zones, Sparse root zones, and Branded Zones. These differ in what gets copied into the zone's root file system, as well as in how they start up. Branded Zones, which starts up using custom scripts and provides special hooks into the operating system, can emulate a different kernel, but that is a topic for another day.

A Full root zone gets a complete copy of all the Solaris installed package files, which needs about 5 GB of storage. A Sparse zone only gets its own /etc and /dev, which saves on space but it uses a read-only loop-back mounted copy of other directories, such as /opt and /usr. Copying all the files into a Full root zone takes some 10 minutes longer than setting up a sparse zone and uses 5 GB more, but because /usr and /opt is writable it results in a more flexible (read more useful) example environment.

Normally before you set up a zone you would need to do thorough planning, taking care of a things like disk space/file systems, CPU and memory resource allocations, maybe even special boot options. But for a quick start lets just assume defaults for most things.

You do need to work out a few details before you start. 1. Select a name for the zone. This can also be the "hostname" for the virtual OS instance in the zone. 2. Select an IP address and identify on which interface it will live. 3. Identify a spot on your file system where the Zone root will be installed.

Lets call the zone "myfirstzone", let it use IP address 192.168.24.131 on interface e1000g0, and give it a directory /export/zones/myfirstzone

mkdir -m 0700 /export/zones/myfirstzone

Yes, you guessed right, later we can add more zones under /export/zones. Be aware, if you don't restrict the permissions on the directory, the zone installation later on will fail with a security error.

The commands for working with zones are:

zonecfg - Change a zone's configuration / setup. This will create and/or modify the xml base configuration of a zone. zoneadm - Manipulate running zones, eg rebooting and showing their status. zlogin - Connect to a zone (Create a shell session in a zone or connect to the zone's console)

Without further ado, creating the zone using zonecfg zonecfg -z myfirstzone zonecfg:myfirstzone> create zonecfg:myfirstzone> set zonepath=/export/zones/myfirstzone zonecfg:myfirstzone> add net zonecfg:myfirstzone:net> set address=192.168.24.131 zonecfg:myfirstzone:net> set physical=e1000g0 zonecfg:myfirstzone:net> end zonecfg:myfirstzone> commit zonecfg:myfirstzone> exit bash-3.2#

The basic Zone has now been set up. We can view it using this command zonecfg -z myfirstzone export create -b set zonepath=/export/zones/myfirstzone set autoboot=false set ip-type=shared add net set address=192.168.24.131 set physical=e1000g0 end

Now to populate it with Solaris packages/files, very simply use the zoneadm install command. This process takes a good 15 minutes or more, and you should see output like this:

bash-3.2# zoneadm -z myfirstzone install Preparing to install zone . Creating list of files to copy from the global zone. Copying <187209> files to the zone. Initializing zone product registry. Determining zone package initialization order. Preparing to initialize <1282> packages on the zone. Initialized <1282> packages on zone. Zone is initialized. Installation of <1> packages was skipped. The file contains a log of the zone installation.

The install process is highly disk IO intensive. If you're building a zone on the same disk from which the system is running, the resulting disk contention can cause it to run quite a bit longer, up to an hour on systems with slow disks.

Use this time to scan through the man pages for the zone commands. There is also an introductory man page, i.e zones(5).

Also while this install is running use the time to look at the output from these commands: zonecfg -z myfirstzone info

zoneadm list -vc

Once the installation process finishes, you can boot the zone and log in. During the first login, you need to login on the zone console and provide the system identification information prompted.

zoneadm -z myfirstzone boot; zlogin -C myfirstzone [Connected to zone 'myfirstzone' console] SunOS Release 5.11 Version snv_80 64-bit Copyright 1983-2007 Sun Microsystems, Inc. All rights reserved. Use is subject to license terms. Hostname: zoneone ... ... ... System identification is completed. rebooting system due to change(s) in /etc/default/init [NOTICE: Zone rebooting] SunOS Release 5.11 Version snv_80 64-bit Copyright 1983-2007 Sun Microsystems, Inc. All rights reserved. Use is subject to license terms. Hostname: zoneone Reading ZFS config: done. zoneone console login:

Now log in as root with the password you just set, and run a few commands in the zone; - Do a ps -ef in the zone, and another one in the Global zone, and compare the output. - Also compare the output from a few other commands, particularly df -h, uptime, zonename, hostname, ifconfig, and uptime. On the ifconfig -a command output, note the zone identifier for each interface, as well as the multiple loopback identifiers.

For now I will leave you to marvel at the differences and similarities, but I will point out one process running in the global zone, namely zoneadmd. It manages the zone environment, ensures that its file systems and network interfaces are mounted and plumbed, etc. There will be an instance of it for every running non-Global zone.

Disconnecting from the zone Console requires that you enter the "disconnect" escape sequence, by default ~. (A tilde followed by a full stop) If you're already working remotely using something like SSH, use ~~. - The extra tilde character will inform SSH that it should not itself act on the disconnect command, but rather pass it on down the server. It is also possible to change your escape sequences, but I'll reserve that for a small blog article on another day. Note that disconnecting the console is NOT the same as logging out of the zones - processes like prstat will continue to run on the zone's console and you can reconnect later by again by running "zlogin -C myfirstzone"

Now that the zone is running, lets do a few interesting things with it.

First, in the Global Zone, have another look at what you see from zoneadm list -vc. Other interesting command are "prstat -Z", as well as "df -Z"

Next try out zlogin sans the -C option which would put you on the console. Use exit, logout, or ^D to disconnect.

Create a new user in the zone zlogin myfirstzone mkdir /export/home useradd -d /export/home/joe -m joe passwd joe

Use SSH to login to the zone from another machine, using the zone's IP address. Confirm where you logged in to using hostname, and zonename.

Now lets start an FTP server in the zone: zlogin myfirstzone svcadm enable ftpd

zlogin as used above runs the specified command directly in the zone. You can check the results using by ftp-ing to the zone, though you need to use a user like joe created above, which can login via the network.

Finally, lets add some disk space to our existing zone. This could have been done during the configuration process, but that would have defeated the purpose of showing how to add something to a zone.

I don't have an extra disk device, so I'll use a loop-back (lofi) device. cd /export/zones mkfile 1g fakedev_for_myfirstzone lofiadm -a `pwd`/fakedev_for_myfirstzone newfs /dev/lofi/1 /dev/rlofi/1: Unable to find Media type. Proceeding with system determined parameters. newfs: construct a new file system /dev/rlofi/1: (y/n)? y /dev/rlofi/1: 2097000 sectors in 3495 cylinders of 1 tracks, 600 sectors 1023.9MB in 219 cyl groups (16 c/g, 4.69MB/g, 2240 i/g) super-block backups (for fsck -F ufs -o b=#) at: 32, 9632, 19232, 28832, 38432, 48032, 57632, 67232, 76832, 86432, 2006432, 2016032, 2025632, 2035232, 2044832, 2054432, 2064032, 2073632, 2083232, 2092832

There are at least 3 ways in which this file system can be added into the domain. I will mount it in the global zone, then loop it into "myfirstzone" (This allws you to mount it into multiple zones should you so wish.)

mkdir mountpoint_for_myfirstzone_space1 mount /dev/lofi/1 `pwd`/mountpoint_for_myfirstzone_space1 zonecfg -z myfirstzone zonecfg:myfirstzone> add fs zonecfg:myfirstzone:fs> set type=lofs zonecfg:myfirstzone:fs> set special=/export/zones/mountpoint_for_myfirstzone_space1 zonecfg:myfirstzone:fs> set dir=/space1 zonecfg:myfirstzone:fs> end zonecfg:myfirstzone> verify zonecfg:myfirstzone> confirm zonecfg:myfirstzone> exit

Make the change take effect zoneadm -z myfirstzone reboot

Log back into the zone, and look at the output of df again. Compare this with the output from df in the global-zone ...

I'll post a follow-up article explaining some of the more advanced zone configuration topics, but hopefully you'll be impressed with how easy this all is. If you want to know why zones work, thing chroot jail combined with ppriv's (man privileges)