Balling’s Bits

Run a QEMU/KVM Virtual Machine in an OmniOS Zone

It is possible to run QEMU/KVM virtual machines inside an OmniOS zone but it requires that the kvm kernel module is made available to the zone. Also, any raw volumes for the virtual machines must be made available to the zone. Here is an example zone configuration file for a QEMU/KVM capable zone:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

create -b
set zonepath=/tank/zones/example
set ip-type=exclusive
set autoboot=true
add net
set physical=dmzexample0
end
add net
set physical=dmzexample1
end
add device
set match=/dev/kvm
end
add device
set match=/dev/zvol/dsk/tank/zones/kvmexample-vol
end
commit

The set match=/dev/kvm makes the kernel kvm module available to the zone and the set match=/dev/zvol/dsk/tank/zones/kvmexample-vol makes the zfs volume tank/zones/kvmexample-vol available in the zone.

The presence of two virtual network interfaces (dmzexample0 and dmzexample1) makes it possible to use one for the zone (and thus VNC into the virtual machine) and the other for the virtual machine.

A sample QEMU/KVM configuration script (/root/kvmexample.sh), in this case for a Plex Media Server, could look like this (ubuntu 14.04 iso mounted in the zone at med/iso via nfs):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

NAME=kvmexample
VNIC0=dmzexample1
HDD0=/dev/zvol/dsk/tank/zones/kvmexample-vol
CD=/med/iso/ubuntu-14.04-server-amd64.iso
VNC=1
MEM=8192

mac0=`dladm show-vnic -po macaddress $VNIC0`

/usr/bin/qemu-system-x86_64 \
-name $NAME \
-boot cd \
-enable-kvm \
-vnc 0.0.0.0:$VNC -k da \
-smp cores=10,threads=1,sockets=1 \
-m $MEM \
-no-hpet \
-localtime \
-drive file=$HDD0,if=virtio,cache=none,index=0 \
-drive file=$CD,media=cdrom,if=ide,index=2  \
-net nic,vlan=40,name=net0,model=virtio,macaddr=$mac0 \
-net vnic,vlan=40,name=net0,ifname=$VNIC0,macaddr=$mac0 \
-vga std \
-pidfile /root/$NAME.pid \
-daemonize

if [ $? -gt 0 ]; then
    echo "Failed to start VM"
fi

port=`expr 5900 + $VNC`
public_nic=$(dladm show-vnic|grep vnic0|awk '{print $2}')
public_ip=$(ifconfig $public_nic|grep inet|awk '{print $2}')

echo "Started VM: $NAME"
echo "VNC available at: host IP ${public_ip} port ${port}"
echo "QEMU Monitor, do: # telnet localhost $TLN. Note: use Control ] to exit monitor before quit!
"

To control (start/stop) the virtual machine as a service using svcadm create a /root/kvmexample.xml manifest (file):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='manifest' name='export'>
    <service name='kvm/kvmexample' type='service' version='0'>
        <create_default_instance enabled='true'/>
        <single_instance/>
        <dependency name='network' grouping='require_all' restart_on='none' type='service'>
            <service_fmri value='svc:/milestone/network:default' />
        </dependency>
        <dependency name='filesystem' grouping='require_all' restart_on='none' type='service'>
            <service_fmri value='svc:/system/filesystem/local:default' />
        </dependency>
        <exec_method name='start' type='method' exec='/root/kvmexample.sh' timeout_seconds='60'/>
        <exec_method name='stop' type='method' exec=':kill' timeout_seconds='60'/>
        <stability value='Unstable'/>
        <template>
            <common_name>
                <loctext xml:lang='C'>KVM-kvmexample</loctext>
            </common_name>
        </template>
    </service>
</service_bundle>

Import the manifest:

1

svccfg import -f kvmexample.xml

And start the virtual machine:

1

svcadm enable kvmexample

Use a VNCviewer to connect to the zones ip-address (on port 5901) to view the display output from the virtual machine.

Stop the virtual machine using:

1

svcadm disable kvmexample

Install Packages From Joyent/SmartOS Repository in OmniOS

There are a few issues with the OmniOS package repositories as mentioned by gea of napp-it:

  • Applications are spread over several repositories
  • Applications are sometimes outdated without regular updates
  • The repositories contain only a few applications
  • The repositories are OS dependent

In contrast, the Joyent/SmartOS package repositories contain a lot of useful packages in one place, and are frequently updated.

To use Joyent/SmartOS packages in OmniOS (here in a zone) start by installing gnu-tar (gtar):

1
2
3
4
5
6
7
8
9
10
11
12
13
14

pkg install gnu-tar
           Packages to install:  1
       Create boot environment: No
Create backup boot environment: No

DOWNLOAD               PKGS       FILES    XFER (MB)
Completed               1/1       41/41      0.9/0.9

PHASE                                        ACTIONS
Install Phase                                123/123

PHASE                                          ITEMS
Package State Update Phase                       1/1
Image State Update Phase                         2/2

Next, install the Joyent/SmartOS bootstrap loader (modify 2014Q2-x86_64 to match current repository and architecture):

1
2
3
4
5

curl http://pkgsrc.joyent.com/packages/SmartOS/bootstrap/bootstrap-2014Q2-x86_64.tar.gz | \
gtar -zxpf - -C /
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 30.9M  100 30.9M    0     0   587k      0  0:00:54  0:00:54 --:--:--  642k

Since Joyent/SmartOS packages are installed in /opt/local update your path (this can be done permanently by modifying the PATH in ~./profile):

1

export PATH=/opt/local/sbin:/opt/local/bin:$PATH

Update the repository database:

1
2
3
4
5
6
7

pkgin -y update
reading local summary...
processing local summary...
updating database: 100%
pkg_summary.bz2                                    100% 1919KB 479.8KB/s  29.3KB/s   00:04
processing remote summary (http://pkgsrc.joyent.com/packages/SmartOS/2014Q2/x86_64/All)...
updating database: 100%

You are ready to search or install packages:

1
2

pkgin search php55-*
pkgin -y install apache-2.4.10

All packages can be viewed at the repository.

Creating and Cloning an OmniOS Zone

An OmniOS/Solaris (non-global) zone acts as completely isolated virtual server within a single operating system instance and shares the kernel with the global zone. It shares resources (CPU and memory) with the global zone and there is close to no overhead in performance. It is an ideal way to isolate different services on a server.

If you are planning on running multiple zones (e.g one for each service), you can save time and resources (disk space) by installing a template zone (base) and then cloning it to make new zones (see end of this post on how to clone an existing zone).

To create a new zone start by creating a zone configuration file (example.conf):

1
2
3
4
5
6
7
8

create -b
set zonepath=/tank/zones/example
set ip-type=exclusive
set autoboot=true
add net
set physical=dmzexample0
end
commit

A ZFS volume will be created at the zonepath /tank/zones/example. The ip-type=exclusive implies that the network stack is separate from the global-zone. The zone will boot upon system boot due to autoboot=true. A single network interface (dmzexample0) is available from within the zone.

Next create the network interface (in the global zone):

1

dladm create-vnic dmzexample0 -l aggr0 -v 40

In this case the dmzexample0 VNIC uses the link aggr0 and is assigned to vlan 40, since the zone is to reside in the DMZ.

Import the zone using zonecfg (in this case the name of the zone will be example):

1

zonecfg -z example -f example.conf

Next install the zone using zoneadm (this takes a couple of minutes):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

zoneadm -z example install
A ZFS file system has been created for this zone.
   Publisher: Using omnios (http://pkg.omniti.com/omnios/release/ ).
   Publisher: Using ms.omniti.com (http://pkg.omniti.com/omniti-ms/).
       Image: Preparing at /tank/zones/example/root.
       Cache: Using /var/pkg/publisher.
Sanity Check: Looking for 'entire' incorporation.
  Installing: Packages (output follows)
           Packages to install: 379
       Create boot environment:  No
Create backup boot environment:  No
            Services to change:   4

DOWNLOAD                                  PKGS       FILES    XFER (MB)
Completed                              379/379 38943/38943  249.4/249.4

PHASE                                        ACTIONS
Install Phase                            56633/56633

PHASE                                          ITEMS
Package State Update Phase                   379/379
Image State Update Phase                         2/2

        Note: Man pages can be obtained by installing pkg:/system/manual
 Postinstall: Copying SMF seed repository ... done.
        Done: Installation completed in 200.260 seconds.

  Next Steps: Boot the zone, then log into the zone console (zlogin -C)
              to complete the configuration process.

Boot the zone and log in using zlogin (to exit the zone just type exit):

1
2
3
4
5

zoneadm -z example boot
zlogin example
[Connected to zone 'example' pts/2]
OmniOS 5.11     006     June 2014
root@example:~#

List available network interfaces:

1
2
3

dladm show-vnic
LINK         OVER         SPEED  MACADDRESS        MACADDRTYPE         VID
dmzexample0  ?            1000   2:8:20:8:51:e9    random              40

Create the IP interface and setup static networking:

1
2
3
4

ipadm create-if dmzexample0
ipadm create-addr -T static -a 192.168.0.10/24 dmzexample0/v4static
# for DHCP use
# ipadm create-addr -T dhcp dmzexample0/v4

Setup routing (if using static networking):

1
2
3

route -p add default 192.168.0.1
# Additional routes can be set up using
# route -p add 10.0.0.0/24 192.168.0.2

Setup name resolution by adding nameservers to /etc/resolv.conf:

1

nameserver 192.168.0.1

Finally configure NSS to use DNS:

1
2

cp /etc/nsswitch.conf{,.bak}
cp /etc/nsswitch.{dns,conf}

Verify internet access:

1
2

ping google.com
google.com is alive

After having successfully installed the zone you might want to take a look at:

To clone an existing zone start by shutting down the zone (from the global zone):

1

zoneadm -z example halt

Copy the configuration file (cp example.conf exampleclone.conf) and modify as appropriate (at least zonepath and physical network):

1
2
3
4
5
6
7
8

create -b
set zonepath=/tank/zones/exampleconf
set ip-type=exclusive
set autoboot=true
add net
set physical=dmzexampleconf0
end
commit

Remember to create any new network interfaces:

1

dladm create-vnic dmzexampleclone0 -l aggr0 -v 40

Import the new configuration as a new zone:

1

zonecfg -z exampleclone -f exampleclone.conf

And then clone the existing example zone using zoneadm and boot it:

1
2

zoneadm -z exampleclone clone example
zoneadm -z exampleclone boot

Next, login and modify network settings (see above).

Finally, to delete a zone start by shutting it down:

1

zoneadm -z exampleclone halt

Then uninstall using zoneadm:

1

zoneadm -z exampleclone uninstall -F

And delete the configuration using zonecfg:

1

zonecfg -z exampleclone delete -F

Make sure the zone no longer appears in the list of zones:

1

zoneadm list -iv