1 of 42

Buildbot Setup for Virtual Machines

This section documents the exact steps used to set up each virtual machine used in the MariaDB Buildbot for building packages and testing. There are lots of boring details, but the information here should be useful for reference, and as a help when installing new virtual images for additional distros and versions.

The articles here document how the virtual machine images were actually set up, not the most efficient way to recreate all of them. Thus, some historical artifacts remain, reflecting the order they were initially added and lessons learned along the way.

For example, the steps for the earliest VMs are somewhat overly enthusiastic about using the qemu-img create -b option to make one vm a copy-on-write copy on top of another image. This makes it harder to modify the images afterwards, since if the base image is modified, any copy-on-write vm on top of it needs to be re-created.

The descriptions do not go into painful detail about the installation procedure. Generally the .iso for the distro was downloaded, and the default, non-graphical installer started. Installing Linux these days is quite easy, and generally just defaults were picked. With respect to harddisk partitioning, generally an 8Gb image was configured, of which 1/2 Gb was reserved for swap and the rest for a single root partition "/".

For details on setting up a host server to run these VMs, see Buildbot Setup for VM host.

Buildbot Setup for Virtual Machines - FreeBSD 9.2

Base install

qemu-img create -f qcow2 /kvm/vms/vm-freebsd92-amd64-serial.qcow2 15G
qemu-img create -f qcow2 /kvm/vms/vm-freebsd92-i386-serial.qcow2 15G

Start each VM booting from the server install iso one at a time and perform the following install steps:

kvm -m 2048 -hda /kvm/vms/vm-freebsd92-amd64-serial.qcow2 -cdrom /kvm/iso/freebsd/9.2/FreeBSD-9.2-RELEASE-amd64-dvd1.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2283-:22
kvm -m 2048 -hda /kvm/vms/vm-freebsd92-i386-serial.qcow2 -cdrom /kvm/iso/freebsd/9.2/FreeBSD-9.2-RELEASE-i386-dvd1.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2284-:22

Once running you can connect to the VNC server from your local host with:

vncviewer -via ${remote_host} localhost

Replace ${remote_host} with the host the vm is running on.

Note: When you activate the install, vncviewer will disconnect after a bit with a complaint about the rect being too large. This is fine. The installer has just resized the vnc screen. Simply reconnect.

Install, picking default options mostly, with the following notes:

Keymap Selection: Do not set non-default key mapping
Hostname: to freebsd-92-amd64 or freebsd-92-i386
Distribution Select: deselect "games", leave "ports" selected
Partitioning: Guided and use the entire disk
- Accept the default partitioning (Finish then Commit)
Archive Extraction: ...wait while installer installs the OS...
Network Configuration: choose vtnet0
- configure ipv4 w/ DHCP
- do not configure ipv6
System Configuration: leave sshd selected, don't select the others
Dumpdev Configuration: do not enable crash dumps
add a buildbot user
- add user to 'wheel' group
Final Configuration: Apply configuration and exit
Manual Configuration: yes to opening a shell
- echo 'console="comconsole"' >> /boot/loader.conf
- Edit /etc/ttys and change off to on and dialup to vt100 for the ttyu0 entry.
- shutdown -p now

Now that the VM is installed, it's time to configure it. If you have the memory you can do the following simultaneously:

kvm -m 2048 -hda /kvm/vms/vm-freebsd92-amd64-serial.qcow2 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2283-:22 -nographic
kvm -m 2048 -hda /kvm/vms/vm-freebsd92-i386-serial.qcow2 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2284-:22 -nographic

ssh -p 2283 localhost
ssh -p 2284 localhost

su -

cd /usr/ports/security/sudo; make install clean # accept defaults
visudo # uncomment the '# %wheel ALL=(ALL) NOPASSWD: ALL' line
exit

cd /usr/ports/ports-mgmt/portmaster; sudo make install clean # select both autocomplete options

ssh hasky.askmonty.org
exit
sudo su buildbot -
ssh hasky.askmonty.org
exit
exit
exit

scp -P 2283 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2283 /kvm/vms/authorized_keys buildbot@localhost:.ssh/

scp -P 2284 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2284 /kvm/vms/authorized_keys buildbot@localhost:.ssh/

ssh -p 2283 localhost
ssh -p 2284 localhost

sudo portsnap fetch extract
sudo portsnap fetch update
sudo portmaster -Da

sudo shutdown -p now

VMs for building BSD bintars

for i in '/kvm/vms/vm-freebsd92-amd64-serial.qcow2 2283 qemu64' '/kvm/vms/vm-freebsd92-i386-serial.qcow2 2284 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
    "cd /usr/ports/devel/libevent;     sudo make install clean BATCH=yes" \
    "cd /usr/ports/devel/boost-all;    sudo make install clean BATCH=yes" \
    "cd /usr/ports/devel/cmake;        sudo make install clean BATCH=yes" \
    "cd /usr/ports/devel/thrift/;      sudo make install clean BATCH=yes" \
    "cd /usr/ports/devel/bzr/;         sudo make install clean BATCH=yes" \
    "cd /usr/ports/devel/bison/;       sudo make install clean BATCH=yes" \
    "cd /usr/ports/databases/unixODBC; sudo make install clean BATCH=yes" \
    "cd /usr/ports/devel/doxygen;      sudo make install clean BATCH=yes"; \
done

VMs for install testing.

qemu-img create -b /kvm/vms/vm-freebsd92-amd64-serial.qcow2 -f qcow2 /kvm/vms/vm-freebsd92-amd64-install.qcow2
qemu-img create -b /kvm/vms/vm-freebsd92-i386-serial.qcow2 -f qcow2 /kvm/vms/vm-freebsd92-i386-install.qcow2

for i in '/kvm/vms/vm-freebsd92-amd64-serial.qcow2 2283 qemu64' '/kvm/vms/vm-freebsd92-i386-serial.qcow2 2284 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
    "sudo portsnap cron update"; \
done

VMs for MySQL upgrade testing

for i in '/kvm/vms/vm-freebsd92-amd64-serial.qcow2 2283 qemu64' '/kvm/vms/vm-freebsd92-i386-serial.qcow2 2284 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
    "cd /usr/ports/databases/mysql55-server;    sudo make install clean BATCH=yes" \
    "sudo /usr/local/etc/rc.d/mysql-server onestart" \
    'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
done

VMs for MariaDB upgrade testing

for i in '/kvm/vms/vm-freebsd92-amd64-serial.qcow2 2283 qemu64' '/kvm/vms/vm-freebsd92-i386-serial.qcow2 2284 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
    "cd /usr/ports/databases/mariadb55-server;    sudo make install clean BATCH=yes" \
    "sudo /usr/local/etc/rc.d/mysql-server onestart" \
    'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"'; \
done

Add Key to known_hosts

Do the following on each kvm host server (terrier, terrier2, i7, etc...) to add the VMs to known_hosts.

# freebsd92-amd64
cp -avi /kvm/vms/vm-freebsd92-amd64-install.qcow2 /kvm/vms/vm-freebsd92-amd64-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-freebsd92-amd64-test.qcow2 -redir tcp:2283::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2283 buildbot@localhost sudo shutdown -p now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-freebsd92-amd64-test.qcow2

# freebsd92-i386
cp -avi /kvm/vms/vm-freebsd92-i386-install.qcow2 /kvm/vms/vm-freebsd92-i386-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-freebsd92-i386-test.qcow2 -redir tcp:2284::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2284 buildbot@localhost sudo shutdown -p now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-freebsd92-i386-test.qcow2

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - General Principles

The installations are kept minimal, picking mostly default options. This helps ensure our stuff works on default installs, and also saves effort during the creation of the virtual machines.

Since most of the logic happens in the Buildbot slave on a host machine, and inside the Buildbot master, there is no need to install Buildbot or other similar complex stuff inside the VMs.

ssh

The virtual machines are configured with ssh access. An account called 'buildbot' is created, with passwordless login (using ssh public key authentication), and with passwordless sudo access (needed for automated scripting). The public key for user buildbot is installed in every VM, and the private key is added to the account running the Buildbot master on the host machine.

To generate the ssh keys:

ssh-keygen -t dsa

Leave the passphrase empty. Run this as the user running the Buildbot slave on the KVM host machine. The resulting /.ssh/id_dsa.pub will be needed below for each virtual machine install.

Serial port

The vms are configured to use the (emulated) serial port as the console. When running KVM on the host, the console then maps to the stdin/stdout. This is useful to get kernel log messages available to easier debug any problems. The bootloader Grub is similarly configured to use the serial port.

The vms are also set up to give a login prompt on the serial port (running getty). In retrospect this has proven not really needed, as if needing to manually log in and investigate things, it is often as easy to just run KVM in graphics mode. It may be useful in cases where the host machine is remote though (in this case graphics mode does still work but can be a bit slow).

To configure for serial console, the following two files are referenced below, and should be prepared in advance:

`ttyS0`

# ttyS0 - getty
#
# This service maintains a getty on ttyS0 from the point the system is
# started until it is shut down again.

start on stopped rc2
start on stopped rc3
start on stopped rc4
start on stopped rc5

stop on runlevel 0
stop on runlevel 1
stop on runlevel 6

respawn
exec /sbin/getty 115200 ttyS0

`ttyS0.conf`

# ttyS0 - getty
#
# This service maintains a getty on ttyS0 from the point the system is
# started until it is shut down again.

start on stopped rc RUNLEVEL=[2345]
stop on runlevel [!2345]

respawn
exec /sbin/getty -L 115200 ttyS0 vt102

Unattended MariaDB install on Debian/Ubuntu

my.seed

The MariaDB (and MySQL) package on Debian and Ubuntu prompts the user about the root password to set. We want to test this important step, but we do not want the prompt of course. To do this we use the following configuration file for defaults for debconf. This file is needed in the steps below under the name "my.seed" (be careful to preserve it exactly as here, including trailing space and tab characters!)

mariadb-server-5.1	mysql-server/root_password_again	password	rootpass
    mariadb-server-5.1	mysql-server/root_password	password	rootpass
    mariadb-server-5.1	mysql-server/error_setting_password	error	
    mariadb-server-5.1	mysql-server-5.1/nis_warning	note	
    mariadb-server-5.1	mysql-server-5.1/really_downgrade	boolean	false
    mariadb-server-5.1	mysql-server-5.1/start_on_boot	boolean	true
    mariadb-server-5.1	mysql-server-5.1/postrm_remove_databases	boolean	false
    mariadb-server-5.1	mysql-server/password_mismatch	error	
    mariadb-server-5.1	mysql-server/no_upgrade_when_using_ndb	error

For some background seehere. The file my.seed can be generated from an existing install using debconf-get-selections.

sources.append

For Debian/Ubuntu, we add a local repository to the list of apt source to be able to test apt-get install. For this, the following file "sources.append" must be prepared:

deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/

Emulation options

The default network card emulated by KVM has poor performance. To solve this we instead use the "virtio" network device, using the KVM options "-net nic,model=virtio -net user". Except on Debian 4, which has an old kernel without support for virtio. Background infohere.

A detail is that some 32-bit vms crashed during boot with default options. This was fixed by using the kvm "-cpu qemu32,-nx" option. Some background information here.

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Additional Steps

This section documents some additions that have been made after the initial setup

Install Cassandra on Fulltest VMs

CassandraSE is no longer actively being developed and has been removed in MariaDB 10.6. See MDEV-23024.

Here are the steps I took to install Cassandra on the Fulltest VMs.

backed up the fulltest VMs with:

rsync -avP /kvm/vms/*fulltest* host:/destination/path/

boot the amd64 fulltest VM:

vm=vm-precise-amd64-fulltest.qcow2
kvm -m 2048 -hda /kvm/vms/${vm} -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:22666-:22 -nographic

ssh -t -p 22666 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i /kvm/vms/ssh-keys/id_dsa dbart@localhost

in the VM, install Cassandra:

sudo vi /etc/apt/sources.list.d/cassandra.list

# paste in the following two lines:
deb http://www.apache.org/dist/cassandra/debian 11x main
deb-src http://www.apache.org/dist/cassandra/debian 11x main

gpg --keyserver pgp.mit.edu --recv-keys F758CE318D77295D
gpg --export --armor F758CE318D77295D | sudo apt-key add -

gpg --keyserver pgp.mit.edu --recv-keys 2B5C1B00
gpg --export --armor 2B5C1B00 | sudo apt-key add -

sudo apt-get update
sudo apt-get install cassandra

in the VM, launch the cassandra-cli program and test the Cassandra installation:

create keyspace DEMO; 
use DEMO; 

create column family Users 
with key_validation_class = 'UTF8Type' 
and comparator = 'UTF8Type' 
and default_validation_class = 'UTF8Type'; 

set Users[1234][name] = scott; 
set Users[1234][password] = tiger; 
get Users[1234]; 
quit;

Output of the above:

dbart@ubuntu-precise-amd64:~$
 cassandra-cli

Connected to: "Test Cluster" on 127.0.0.1/9160
Welcome to Cassandra CLI version 1.1.9

Type 'help;' or '?' for help.
Type 'quit;' or 'exit;' to quit.

[default@unknown]
 create keyspace DEMO;

622a672f-dd03-37bf-bf78-f3e99a8f18a6
Waiting for schema agreement...
... schemas agree across the cluster
[default@unknown]
 use DEMO;

Authenticated to keyspace: DEMO
[default@DEMO]
 create column family Users

...    
 with key_validation_class = 'UTF8Type'

...    
 and comparator = 'UTF8Type'

...    
 and default_validation_class = 'UTF8Type';

605eea14-d3e5-3d1d-ab1d-f4863c814538
Waiting for schema agreement...
... schemas agree across the cluster
[default@DEMO]
 set Users[1234][name] = scott;

Value inserted.
Elapsed time: 46 msec(s).
[default@DEMO]
 set Users[1234][password] = tiger;

Value inserted.
Elapsed time: 2.77 msec(s).
[default@DEMO]
 get Users[1234];

=> (column=name, value=scott, timestamp=1361818884084000)
=> (column=password, value=tiger, timestamp=1361818887944000)
Returned 2 results.
Elapsed time: 53 msec(s).
[default@DEMO]
 quit;

in the VM, shut it down:

sudo shutdown -h now

Do steps 2-6 for vm-precise-i386-fulltest.qcow2. The output of the testing step was:

dbart@ubuntu-precise-i386:~$
 cassandra-cli

Connected to: "Test Cluster" on 127.0.0.1/9160
Welcome to Cassandra CLI version 1.1.9

Type 'help;' or '?' for help.
Type 'quit;' or 'exit;' to quit.

[default@unknown]
 create keyspace DEMO;

5eafc25e-71b6-3585-9db1-891b3348790c
Waiting for schema agreement...
... schemas agree across the cluster
[default@unknown]
 use DEMO;

Authenticated to keyspace: DEMO
[default@DEMO]
 create column family Users

...    
 with key_validation_class = 'UTF8Type'

...    
 and comparator = 'UTF8Type'

...    
 and default_validation_class = 'UTF8Type';

9c2ad7bc-8dc0-35ce-8067-4dc4577319f1
Waiting for schema agreement...
... schemas agree across the cluster
[default@DEMO]
 set Users[1234][name] = scott;

Value inserted.
Elapsed time: 51 msec(s).
[default@DEMO]
 set Users[1234][password] = tiger;

Value inserted.
Elapsed time: 2.44 msec(s).
[default@DEMO]
 get Users[1234];

=> (column=name, value=scott, timestamp=1361819341068000)
=> (column=password, value=tiger, timestamp=1361819345337000)
Returned 2 results.
Elapsed time: 57 msec(s).
[default@DEMO]
 quit;

on the other build hosts, rsync the files over:

rsync -avP host::kvm/vms/*fulltest* /kvm/vms/

_{This page is licensed: CC BY-SA / Gnu FDL}

Install cmake on build VMs

MariaDB 5.5 Requires cmake. Install cmake on all -build VMs (and other Unix-like machines) with:

wget http://www.cmake.org/files/v2.8/cmake-2.8.8.tar.gz
tar -zxvf cmake-2.8.8.tar.gz
cd cmake-2.8.8;./configure
make
sudo make install

_{This page is licensed: CC BY-SA / Gnu FDL}

Installing correct libraries for PAM and readline

Some additional/alternative libraries needs to be installed to handle readline and PAM correctly.

The newer libreadline is GPLv3 and so not compatible with the MariaDB/MySQL GPLv2 license. The PAM libraries are needed for the PAM plugin.

On the Centos and RHEL -build VMs, install the pam-devel package:

sudo yum install pam-devel

On all the Debian/Ubuntu -build virtual machines, install libpam0g-dev:

sudo apt-get install libpam0g-dev

On debian6/maverick/natty, install libreadline5-dev (replacing any libreadline6-dev already there):

sudo apt-get install libreadline5-dev

On oneiric (and any newer, eg. Debian 7 or Ubuntu 12.04), the package is called libreadline-gplv2-dev:

sudo apt-get install libreadline-gplv2-dev

_{This page is licensed: CC BY-SA / Gnu FDL}

Installing the Boost library needed for the OQGraph storage engine

The OQGraph storage engine needs a newer version of Boost that what is available on (most) distributions. The version installed is 1.42.0, available from.

The boost library is installed in all the builder virtual machine images with the following single command:

for i in "vm-hardy-amd64-build qemu64" "vm-hardy-i386-build qemu32,-nx" \
        "vm-intrepid-amd64-build qemu64" "vm-intrepid-i386-build qemu32,-nx" \
        "vm-karmic-amd64-build qemu64" "vm-karmic-i386-build qemu32,-nx" \
        "vm-jaunty-amd64-build qemu64" "vm-jaunty-i386-deb-build qemu32,-nx" \
        "vm-lucid-amd64-build qemu64" "vm-lucid-i386-build qemu32,-nx" \
        "vm-maverick-amd64-build qemu64" "vm-maverick-i386-build qemu32,-nx" \
        "vm-natty-amd64-build qemu64" "vm-natty-i386-build qemu64" \
        "vm-oneiric-amd64-build qemu64" "vm-oneiric-i386-build qemu64" \
        "vm-debian5-amd64-build qemu64" "vm-debian5-i386-build qemu32,-nx" \
        "vm-debian4-amd64-build qemu64 --netdev=e1000" "vm-debian4-i386-build qemu32,-nx --netdev=e1000" \
        "vm-centos5-i386-build qemu32,-nx" "vm-centos5-amd64-build qemu64" \
        "vm-hardy-amd64-build qemu64" "vm-hardy-i386-build qemu32,-nx" \
        "vm-jaunty-i386-deb-tarbake qemu32,-nx" ; do \
  set $i; \
  runvm -m 512 --smp=1 --port=2200 --user=buildbot --cpu=$2 $3 $1.qcow2 \
    "= scp -P 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/boost_1_42_0.tar.gz buildbot@localhost:/dev/shm/" \
    "sudo mkdir -p /usr/local/src /usr/local/include" \
    "cd /usr/local/src && sudo tar zxf /dev/shm/boost_1_42_0.tar.gz" \
    "cd /usr/local/include && sudo ln -s ../src/boost_1_42_0/boost ." ; \
done

Upgrade Boost to 1.49

To upgrade boost on the VMs to 1.49 I performed the following steps:

I initially tried to upgrade the VMs using a script like the one at the top of the page which was used to install boost 1.42, but I ran into issues with getting it to work on all of the VMs (it worked on some, but not on others). So I ended up using the steps below.

Copy the VM (keeping the original as a backup, in case something goes wrong):

oldvm="vm-debian6-i386-build.qcow2"
newvm="vm-debian6-i386-build.upd.qcow2"
cp -avi ${oldvm} ${newvm}

Start the VM:

kvm -m 1024 -hda /kvm/vms/${newvm} -redir 'tcp:22775::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic

Copy the boost tar.gz file over to the VM:

scp -i /kvm/vms/ssh-keys/id_dsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -P 22775 /kvm/boost_1_49_0.tar.gz buildbot@localhost:/dev/shm/

ssh into the VM:

ssh -i /kvm/vms/ssh-keys/id_dsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -p 22775 buildbot@localhost

once inside the VM, perform the following steps:

cd /usr/local/src
sudo tar zxf /dev/shm/boost_1_49_0.tar.gz 
cd /usr/local/include/
sudo rm boost
sudo ln -s ../src/boost_1_49_0/boost .
sudo /sbin/shutdown -h now

_{This page is licensed: CC BY-SA / Gnu FDL}

Installing VM images for testing .deb upgrade between versions

This step creates virtual machine images used to do an important additional upgrade test for .debs. Each virtual machine is pre-installed with an older version of MariaDB, and the test tries upgrading this version to the newly build one (to check that dependencies etc. work out correctly).

This step may not be easily possible to replicate exactly for new or updated images. For example, when we add a new platform/distro, we will not have the same old version of MariaDB available for installation on the new planform. Or if needing to re-create an image in the future, the original old .debs used before may no longer be available. However, this is not a big problem, as we can just use whatever version of MariaDB is available. In fact, while we cannot reasonably test every possible upgrade combination between MariaDB versions, it is still useful to test upgrades from different versions on different platforms, to increase coverage a bit.

The bulk of the images were installed with the following two loops, using packages from the OurDelta repository for MariaDB 5.1.42, which was the one available from there at the time of installation.

for i in "vm-hardy-amd64-install qemu64 hardy" "vm-hardy-i386-install qemu32,-nx hardy" \
          "vm-intrepid-amd64-install qemu64 intrepid" "vm-intrepid-i386-install qemu32,-nx intrepid" \
          "vm-karmic-amd64-install qemu64 karmic" "vm-karmic-i386-install qemu32,-nx karmic" \
          "vm-jaunty-amd64-install qemu64 jaunty" "vm-jaunty-i386-deb-install qemu32,-nx jaunty" \
          "vm-debian5-amd64-install qemu64 lenny" "vm-debian5-i386-install qemu32,-nx lenny" ; do \
  set $i; \
  runvm -b  $1.qcow2 -m 512 --smp=1 --port=2200 --user=buildbot --cpu=$2 $(echo $1 | sed -e 's/-install/-upgrade2/').qcow2 \
    "wget -O- http://ourdelta.org/deb/ourdelta.gpg | sudo apt-key add -" \
    "sudo sh -c \"echo 'deb http://mirror.ourdelta.org/deb $3 mariadb-ourdelta' > /etc/apt/sources.list.d/ourdelta.list\"" \
    "sudo apt-get update || true" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mariadb-server mariadb-test libmariadbclient-dev || true" \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
    "sudo rm /etc/apt/sources.list.d/ourdelta.list" \
    "sudo apt-get update || true" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y || true" ; \
done

for i in "vm-debian4-amd64-install qemu64 etch /kvm/debian-40r8-amd64-netinst.iso" "vm-debian4-i386-install qemu32,-nx etch /kvm/debian-40r8-i386-netinst.iso" ; do \
  set $i; \
  runvm -b  $1.qcow2 -m 512 --smp=1 --port=2200 --user=buildbot --cpu=$2 --netdev=e1000 --kvm=-cdrom --kvm=$4 $(echo $1 | sed -e 's/-install/-upgrade2/').qcow2 \
    "wget -O- http://ourdelta.org/deb/ourdelta.gpg | sudo apt-key add -" \
    "sudo sh -c \"echo 'deb http://mirror.ourdelta.org/deb $3 mariadb-ourdelta' > /etc/apt/sources.list.d/ourdelta.list\"" \
    "sudo apt-get update || true" \
    "sudo sh -c 'DEBIAN_FRONTEND=noninteractive apt-get install -y mariadb-server mariadb-test libmariadbclient-dev' || true" \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
    "sudo rm /etc/apt/sources.list.d/ourdelta.list" \
    "sudo apt-get update || true" \
    "sudo sh -c 'DEBIAN_FRONTEND=noninteractive apt-get upgrade -y' || true" ; \
done

The Ubuntu 10.04 "lucid" images were installed manually (as no packages for lucid were available from OurDelta at the time of installation):

Create and boot 64-bit lucid upgrade image:

qemu-img create -b vm-lucid-amd64-install.qcow2 -f qcow2 vm-lucid-amd64-upgrade2.qcow2 
kvm -m 512 -hda vm-lucid-amd64-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic

Create directory inside the image:

mkdir buildbot

Copy in packages to be installed from outside:

scp -P 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r /archive/pack/mariadb-5.1-knielsen/build-277/kvm-deb-lucid-amd64/debs buildbot@localhost:buildbot/

Install the MariaDB packages, remove package dir, and upgrade to latest security fixes:

sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
rm -Rf buildbot
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y

Now the same, but for 32-bit lucid:

qemu-img create -b vm-lucid-i386-install.qcow2 -f qcow2 vm-lucid-i386-upgrade2.qcow2 
kvm -m 512 -hda vm-lucid-i386-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

Create directory inside the image:

mkdir buildbot

Copy in packages to be installed from outside:

scp -P 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r /archive/pack/mariadb-5.1-knielsen/build-277/kvm-deb-lucid-x86/debs buildbot@localhost:buildbot/

Install the MariaDB packages, remove package dir, and upgrade to latest security fixes:

sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
rm -Rf buildbot
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y

The Ubuntu 10.10 "maverick" images were likewise installed manually:

Create and boot 64-bit maverick upgrade image:

qemu-img create -b vm-maverick-amd64-install.qcow2 -f qcow2 vm-maverick-amd64-upgrade2.qcow2
kvm -m 512 -hda vm-maverick-amd64-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic

ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost mkdir buildbot
scp -P 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r /archive/pack/mariadb-5.1-knielsen/build-619/kvm-deb-maverick-amd64/debs buildbot@localhost:buildbot/
ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
rm -Rf buildbot
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y

Same for 32-bit maverick:

qemu-img create -b vm-maverick-i386-install.qcow2 -f qcow2 vm-maverick-i386-upgrade2.qcow2
kvm -m 512 -hda vm-maverick-i386-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost mkdir buildbot
scp -P 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r /archive/pack/mariadb-5.1-knielsen/build-619/kvm-deb-maverick-x86/debs buildbot@localhost:buildbot/
ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
rm -Rf buildbot
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y

For Ubuntu 11.04 "natty", mariadb packages were installed from the repository for the previous version.

64-bit Ubuntu natty:

qemu-img create -b vm-natty-amd64-install.qcow2 -f qcow2 vm-natty-amd64-upgrade2.qcow2
kvm -m 512 -hda vm-natty-amd64-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

sudo sh -c 'cat > /etc/apt/sources.list.d/tmp.list'
deb http://ftp.osuosl.org/pub/mariadb/repo/5.1/ubuntu maverick main
deb-src http://ftp.osuosl.org/pub/mariadb/repo/5.1/ubuntu maverick main

sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
sudo rm /etc/apt/sources.list.d/tmp.list
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y

32-bit Ubuntu natty:

qemu-img create -b vm-natty-i386-install.qcow2 -f qcow2 vm-natty-i386-upgrade2.qcow2
kvm -m 512 -hda vm-natty-i386-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

sudo sh -c 'cat > /etc/apt/sources.list.d/tmp.list'
deb http://ftp.osuosl.org/pub/mariadb/repo/5.1/ubuntu maverick main
deb-src http://ftp.osuosl.org/pub/mariadb/repo/5.1/ubuntu maverick main

sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
sudo rm /etc/apt/sources.list.d/tmp.list
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y

For Ubuntu 11.10 "oneiric". The steps performed are listed at: Buildbot Setup for Virtual Machines - Ubuntu 11.10 "oneiric" (in the VMs for Upgrade Testing section)

_{This page is licensed: CC BY-SA / Gnu FDL}

Resizing a Virtual Machine Image

Some KVM images end up not having enough space on them. In such cases, it is preferable to increase the size of the VM rather than to just delete an image and rebuild it from scratch. The steps outlined below document what was done to increase the size of the Red Hat 5 x86 VM and should be able to be easily adapted to other VMs, should they need the same treatment in the future.

Make a copy of the VM to work on (we don't want to change the original):

cp -avi vm-rhel5-x86-build.qcow2 vm-rhel5-x86-build-new.qcow2

Using the qemu-img command, resize the image:

qemu-img info vm-rhel5-x86-build-new.qcow2
qemu-img resize vm-rhel5-x86-build-new.qcow2 +10G
qemu-img info vm-rhel5-x86-build-new.qcow2
rsync -avP vm-rhel5-x86-build-new.qcow2 terrier:/kvm/vms/

Not all versions of qemu-img can resize VMs.

Boot the newly resized image with gparted:

vm=vm-rhel5-x86-build-new.qcow2
kvm -m 2048 -hda /kvm/vms/${vm} -cdrom /kvm/iso/gparted-live-0.14.1-6-i486.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:22666-:22

Connect to the VM using VNC from your local machine:

vncviewer -via <vmhost> localhost

Midway through booting you'll have to reconnect

Use gparted to either expand the existing primary partition or, especially on VMs with LVM, add a new partition (since GParted can't change LVM partitions). Exit when finished and shutdown the VM.
Boot the VM again, this time without a VNC server:

kvm -m 2048 -hda /kvm/vms/${vm} -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:22666-:22 -nographic

ssh -t -p 22666 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa buildbot@localhost

if expanded an existing partition: verify the new size
else if you created a new partition:

mount the new partition to tmp location

sudo mount /dev/hda3 /mnt

rsync contents of /home/ to the new partition

sudo rsync -avP /home/ /mnt/

edit fstab to mount new partition to /home

sudo vi /etc/fstab

mv /home to /home-old, create /home dir, mount /home

sudo mv -vi /home /home-old;sudo mkdir -v /home;sudo mount /home

(optional) unmount /mnt

sudo umount /mnt

reboot and verify that things look good

sudo /sbin/shutdown -h now

if things do look good (new drive mounted OK, accounts work, etc...), delete /home-old

Move the old VM to -old and the -new VM to what the old VM used to be named

sudo mv -vi /kvm/vms/vm-rhel5-x86-build.qcow2 vm-rhel5-x86-build-old.qcow2; sudo mv -vi /kvm/vms/vm-rhel5-x86-build-new.qcow2 /kvm/vms/vm-rhel5-x86-build.qcow2

on other VM hosts, make a copy of the old file then rsync over the updated files (the copy helps speed up the rsync):

sudo cp -avi /kvm/vms/vm-rhel5-x86-build.qcow2 /kvm/vms/vm-rhel5-x86-build-old.qcow2
sudo rsync -avP terrier.askmonty.org::kvm/vms/vm-rhel5-x86-build* /kvm/vms/

Test the new VM with a build to make sure it works

_{This page is licensed: CC BY-SA / Gnu FDL}

Update Debian 4 mirrors for Buildbot VMs

Debian 4 has become so old that the apt repository has been moved out of the main Debian 4 mirror servers, and into the archive of old versions. This needs to be fixed by pointing the Debian 4 images to a different mirror:

64-bit:

kvm -m 512 -hda vm-debian4-amd64-install.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=e1000 -net user -nographic

sudo vi /etc/apt/sources.list
# replace http://ftp.dk.debian.org/debian/ with http://ftp.de.debian.org/archive/debian/

32-bit:

kvm -m 512 -hda vm-debian4-i386-install.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=e1000 -net user -nographic

sudo vi /etc/apt/sources.list
# replace http://ftp.dk.debian.org/debian/ with http://ftp.de.debian.org/archive/debian/

After that, it is necessary to re-do from scratch the -update and -update2 debian4 images (as these are built on top of the -install images).

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Debian

Buildbot Setup for Virtual Machines - Debian 4 amd64

cd /kvm/vms
qemu-img create -f qcow2 vm-debian4-amd64-serial.qcow2 8G
kvm -m 2047 -hda /kvm/vms/vm-debian4-amd64-serial.qcow2 -cdrom /kvm/debian-40r8-amd64-netinst.iso -redir 'tcp:2240::22' -boot d -smp 2 -cpu qemu64 -net nic,model=e1000 -net user

Serial console and account setup

From base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:

kvm -m 2047 -hda /kvm/vms/vm-debian4-amd64-serial.qcow2 -cdrom /kvm/debian-40r8-amd64-netinst.iso -redir 'tcp:2240::22' -boot c -smp 2 -cpu qemu64 -net nic,model=e1000 -net user

su
apt-get install sudo openssh-server
VISUAL=vi visudo
# Add at the end:t %sudo ALL=NOPASSWD: ALL

# Add account <USER> to group sudo
# Copy in public ssh key.

# Add in /etc/inittab:
S0:2345:respawn:/sbin/getty -L ttyS0 19200 vt100

Add to /boot/grub/menu.lst:

serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial console

also add in menu.lst to kernel line:

console=tty0 console=ttyS0,115200n8

# Add user buildbot, with disabled password. Add as sudo, and add ssh key.
sudo /usr/sbin/adduser --disabled-password buildbot
sudo /usr/sbin/adduser buildbot sudo
sudo su - buildbot
mkdir .ssh
# Add all necessary keys.
cat >.ssh/authorized_keys
chmod -R go-rwx .ssh

VM for build

qemu-img create -b vm-debian4-amd64-serial.qcow2 -f qcow2 vm-debian4-amd64-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian4-amd64-build.qcow2 -cdrom /kvm/debian-40r8-amd64-netinst.iso -redir 'tcp:2240::22' -boot c -smp 2 -cpu qemu64 -net nic,model=e1000 -net user -nographic

sudo apt-get build-dep mysql-server-5.0
# Some latex packages fail to install because they complain that the
# source is more than 5 years old! I solved by setting back the clock a
# couple of years temporarily ...
sudo apt-get install devscripts doxygen texlive-latex-base gs lsb-release fakeroot libevent-dev libssl-dev zlib1g-dev libreadline5-dev

VM for install testing

qemu-img create -b vm-debian4-amd64-serial.qcow2 -f qcow2 vm-debian4-amd64-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian4-amd64-install.qcow2 -cdrom /kvm/debian-40r8-amd64-netinst.iso -redir 'tcp:2240::22' -boot c -smp 2 -cpu qemu64 -net nic,model=e1000 -net user -nographic

See the General Principles article for how to make the 'my.seed' file.

# No packages mostly!
sudo apt-get install debconf-utils

cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END

sudo debconf-set-selections /tmp/my.seed

VM for upgrade testing

qemu-img create -b vm-debian4-amd64-install.qcow2 -f qcow2 vm-debian4-amd64-upgrade.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian4-amd64-upgrade.qcow2 -cdrom /kvm/debian-40r8-amd64-netinst.iso -redir 'tcp:2240::22' -boot c -smp 2 -cpu qemu64 -net nic,model=e1000 -net user -nographic

sudo apt-get install mysql-server-5.0
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Debian 4 i386

Create the VM:

cd /kvm/vms
qemu-img create -f qcow2 vm-debian4-i386-serial.qcow2 8G
kvm -m 2047 -hda /kvm/vms/vm-debian4-i386-serial.qcow2 -cdrom /kvm/debian-40r8-i386-netinst.iso -redir 'tcp:2241::22' -boot d -smp 2 -cpu qemu32,-nx -net nic,model=e1000 -net user

Serial console and account setup

From base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:

kvm -m 2047 -hda /kvm/vms/vm-debian4-i386-serial.qcow2 -cdrom /kvm/debian-40r8-i386-netinst.iso -redir 'tcp:2241::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=e1000 -net user

su
apt-get install sudo openssh-server
VISUAL=vi visudo

# Add at the end:t %sudo ALL=NOPASSWD: ALL

# Add account <USER> to group sudo
# Copy in public ssh key.

# Add in /etc/inittab:
S0:2345:respawn:/sbin/getty -L ttyS0 19200 vt100

Add to /boot/grub/menu.lst:

serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial console

also add in menu.lst to kernel line:

console=tty0 console=ttyS0,115200n8

Do these steps:

# Add user buildbot, with disabled password. Add as sudo, and add ssh key.
sudo /usr/sbin/adduser --disabled-password buildbot
sudo /usr/sbin/adduser buildbot sudo
sudo su - buildbot
mkdir .ssh
# Add all necessary keys.
cat >.ssh/authorized_keys
chmod -R go-rwx .ssh

VM for build

qemu-img create -b vm-debian4-i386-serial.qcow2 -f qcow2 vm-debian4-i386-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian4-i386-build.qcow2 -cdrom /kvm/debian-40r8-i386-netinst.iso -redir 'tcp:2241::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=e1000 -net user -nographic

sudo apt-get build-dep mysql-server-5.0
# Some latex packages fail to install because they complain that the
# source is more than 5 years old! I solved by setting back the clock a
# couple of years temporarily ...
sudo apt-get install devscripts doxygen texlive-latex-base gs lsb-release fakeroot libevent-dev libssl-dev zlib1g-dev libreadline5-dev

VM for install testing

qemu-img create -b vm-debian4-i386-serial.qcow2 -f qcow2 vm-debian4-i386-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian4-i386-install.qcow2 -cdrom /kvm/debian-40r8-i386-netinst.iso -redir 'tcp:2241::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=e1000 -net user -nographic

See the General Principles article for how to make the 'my.seed' file.

# No packages mostly!
sudo apt-get install debconf-utils

cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END

sudo debconf-set-selections /tmp/my.seed

VM for upgrade testing

qemu-img create -b vm-debian4-i386-install.qcow2 -f qcow2 vm-debian4-i386-upgrade.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian4-i386-upgrade.qcow2 -cdrom /kvm/debian-40r8-i386-netinst.iso -redir 'tcp:2241::22' -boot c -smp 2 -cpu qemu64 -net nic,model=e1000 -net user -nographic

sudo apt-get install mysql-server-5.0
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Debian 5 amd64

Download netinst CD image debian-503-amd64-netinst.iso

cd /kvm/vms
qemu-img create -f qcow2 vm-debian5-amd64-serial.qcow2 8G
kvm -m 2047 -hda /kvm/vms/vm-debian5-amd64-serial.qcow2 -cdrom /kvm/debian-503-amd64-netinst.iso -redir 'tcp:2234::22' -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Serial console and account setup

From base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:

kvm -m 2047 -hda /kvm/vms/vm-debian5-amd64-serial.qcow2 -redir 'tcp:2234::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user

su
apt-get install sudo openssh-server
visudo
# uncomment %sudo ALL=NOPASSWD: ALL
# add user account to group sudo `addgroup <USER> sudo`.
# Copy in public ssh key.

# Add in /etc/inittab:
S0:2345:respawn:/sbin/getty -L ttyS0 19200 vt100

Add to /boot/grub/menu.lst:

serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial console

also add in menu.lst to kernel line (after removing `quiet splash'):

console=tty0 console=ttyS0,115200n8

Add user buildbot, with disabled password. Add as sudo, and add ssh key.

sudo adduser --disabled-password buildbot
sudo adduser buildbot sudo
sudo su - buildbot
mkdir .ssh
# Add all necessary keys.
cat >.ssh/authorized_keys
chmod -R go-rwx .ssh

VM for building .deb

qemu-img create -b vm-debian5-amd64-serial.qcow2 -f qcow2 vm-debian5-amd64-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian5-amd64-build.qcow2 -redir 'tcp:2234::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

sudo apt-get build-dep mysql-server-5.0
sudo apt-get install devscripts hardening-wrapper doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline5-dev

VM for install testing

qemu-img create -b vm-debian5-amd64-serial.qcow2 -f qcow2 vm-debian5-amd64-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian5-amd64-install.qcow2 -redir 'tcp:2234::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

# No packages mostly!
sudo apt-get install debconf-utils

cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END

sudo debconf-set-selections /tmp/my.seed

See the General Principles article for how to make the 'my.seed' file.

VM for upgrade testing

qemu-img create -b vm-debian5-amd64-install.qcow2 -f qcow2 vm-debian5-amd64-upgrade.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian5-amd64-upgrade.qcow2 -redir 'tcp:2234::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

sudo apt-get install mysql-server-5.0
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Debian 5 i386

Base install

Download netinst CD image debian-503-i386-netinst.iso and install:

cd /kvm
qemu-img create -f qcow2 vms/vm-debian5-i386-base.qcow2 8G
kvm -m 2047 -hda /kvm/vms/vm-debian5-i386-base.qcow2 -cdrom /kvm/debian-503-i386-netinst.iso -redir 'tcp:2226::22' -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user

Serial port console

From base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:

qemu-img create -b vm-debian5-i386-base.qcow2 -f qcow2 vm-debian5-i386-serial.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian5-i386-serial.qcow2 -redir 'tcp:2226::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
su
apt-get install sudo openssh-server emacs22-nox
visudo
# uncomment %sudo ALL=NOPASSWD: ALL
# add user account to group sudo.
# Copy in public ssh key.

# Add in /etc/inittab:
S0:2345:respawn:/sbin/agetty -h -L ttyS0 19200 vt100

Add to /boot/grub/menu.lst:

serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial console

also add in menu.lst to kernel line (after removing `quiet splash'):

console=tty0 console=ttyS0,115200n8

Add user buildbot, with disabled password. Add as sudo, and add ssh key.

sudo adduser --disabled-password buildbot
sudo adduser buildbot sudo
sudo su - buildbot
mkdir .ssh
# Paste all necessary keys.
cat >.ssh/authorized_keys
chmod -R go-rwx .ssh

Image for .deb build

qemu-img create -b vm-debian5-i386-serial.qcow2 -f qcow2 vm-debian5-i386-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian5-i386-build.qcow2 -redir 'tcp:2226::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

sudo apt-get build-dep mysql-server-5.0
sudo apt-get install devscripts hardening-wrapper doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline5-dev

Image for install testing

qemu-img create -b vm-debian5-i386-serial.qcow2 -f qcow2 vm-debian5-i386-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian5-i386-install.qcow2 -redir 'tcp:2226::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

# No packages mostly!
sudo apt-get install debconf-utils

cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END

sudo debconf-set-selections /tmp/my.seed

See the General Principles article how to obtain the my.seed file.

Image for upgrade testing

qemu-img create -b vm-debian5-i386-install.qcow2 -f qcow2 vm-debian5-i386-upgrade.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian5-i386-upgrade.qcow2 -redir 'tcp:2226::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

sudo apt-get install mysql-server-5.0
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Debian 6 amd64

Create the VM:

cd /kvm/vms
qemu-img create -f qcow2 vm-debian6-amd64-serial.qcow2 8G
kvm -m 2047 -hda /kvm/vms/vm-debian6-amd64-serial.qcow2 -cdrom /kvm/debian-6a1-amd64-netinst.iso -redir 'tcp:2244::22' -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Serial console and account setup

From base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:

kvm -m 2047 -hda /kvm/vms/vm-debian6-amd64-serial.qcow2 -redir 'tcp:2244::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user

su
apt-get install sudo openssh-server
visudo

# Add: %sudo ALL=NOPASSWD: ALL
# add user account to group sudo `addgroup <USER> sudo`.
# Copy in public ssh key.

# Add in /etc/inittab:
S0:2345:respawn:/sbin/getty -L ttyS0 19200 vt100

Editing /etc/default/grub:

sudo vi /etc/default/grub
# Add/edit these entries:
    GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
    GRUB_TERMINAL="serial"
    GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

sudo update-grub

Add user buildbot, with disabled password. Add as sudo, and add ssh key.

sudo adduser --disabled-password buildbot
sudo adduser buildbot sudo
sudo su - buildbot
mkdir .ssh
# Add all necessary keys.
cat >.ssh/authorized_keys
chmod -R go-rwx .ssh

VM for building .deb

qemu-img create -b vm-debian6-amd64-serial.qcow2 -f qcow2 vm-debian6-amd64-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian6-amd64-build.qcow2 -redir 'tcp:2244::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

sudo apt-get build-dep mysql-server-5.1
sudo apt-get install devscripts hardening-wrapper doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline-dev

VM for install testing

qemu-img create -b vm-debian6-amd64-serial.qcow2 -f qcow2 vm-debian6-amd64-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian6-amd64-install.qcow2 -redir 'tcp:2244::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

See the General Principles article for how to make the 'my.seed' file.

# No packages mostly!
sudo apt-get install debconf-utils

cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END

sudo debconf-set-selections /tmp/my.seed

VM for upgrade testing

qemu-img create -b vm-debian6-amd64-install.qcow2 -f qcow2 vm-debian6-amd64-upgrade.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian6-amd64-upgrade.qcow2 -redir 'tcp:2244::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

sudo apt-get install mysql-server-5.1
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Debian 6 i386

Base Install

cd /kvm/vms
qemu-img create -f qcow2 vm-debian6-i386-serial.qcow2 8G
kvm -m 2047 -hda /kvm/vms/vm-debian6-i386-serial.qcow2 -cdrom /kvm/debian-6a1-i386-netinst.iso -redir 'tcp:2245::22' -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Serial console and account setup

From base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:

kvm -m 2047 -hda /kvm/vms/vm-debian6-i386-serial.qcow2 -redir 'tcp:2245::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user

su
apt-get install sudo openssh-server
visudo
# Add: %sudo ALL=NOPASSWD: ALL
# add user account to group sudo `addgroup <USER> sudo`.
# Copy in public ssh key.

# Add in /etc/inittab:
S0:2345:respawn:/sbin/getty -L ttyS0 19200 vt100

Editing /etc/default/grub:

sudo vi /etc/default/grub

# Add/edit these entries:
    GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
    GRUB_TERMINAL="serial"
    GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

sudo update-grub

Add user buildbot, with disabled password. Add as sudo, and add ssh key.

sudo adduser --disabled-password buildbot
sudo adduser buildbot sudo
sudo su - buildbot
mkdir .ssh
# Add all necessary keys.
cat >.ssh/authorized_keys
chmod -R go-rwx .ssh

VM for building .deb

qemu-img create -b vm-debian6-i386-serial.qcow2 -f qcow2 vm-debian6-i386-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian6-i386-build.qcow2 -redir 'tcp:2245::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

sudo apt-get build-dep mysql-server-5.1
sudo apt-get install devscripts hardening-wrapper doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline-dev

VM for install testing

qemu-img create -b vm-debian6-i386-serial.qcow2 -f qcow2 vm-debian6-i386-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian6-i386-install.qcow2 -redir 'tcp:2245::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

See above for how to obtain my.seed

# No packages mostly!
sudo apt-get install debconf-utils

cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END

sudo debconf-set-selections /tmp/my.seed

VM for upgrade testing

qemu-img create -b vm-debian6-i386-install.qcow2 -f qcow2 vm-debian6-i386-upgrade.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian6-i386-upgrade.qcow2 -redir 'tcp:2245::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

sudo apt-get install mysql-server-5.1
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Debian 7 "wheezy"

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Red Hat

This category contains articles documenting how the MariaDB Red Hat-based buildbot build and test VMs were created

Buildbot Setup for Virtual Machines - Centos 5 amd64

Base install

cd /kvm
wget http://ftp.klid.dk/ftp/centos/5.3/isos/x86_64/CentOS-5.3-x86_64-bin-DVD.iso
qemu-img create -f qcow2 vms/vm-centos5-amd64-serial.qcow2 8G
kvm -m 2047 -hda /kvm/vms/vm-centos5-amd64-serial.qcow2 -cdrom CentOS-5.3-x86_64-bin-DVD.iso -redir 'tcp:2237::22' -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Serial port and account setup

kvm -m 2047 -hda /kvm/vms/vm-centos5-amd64-serial.qcow2 -cdrom CentOS-5.3-i386-bin-DVD.iso -redir 'tcp:2237::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

Add to /boot/grub/menu.lst:

serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial console

also add in menu.lst to kernel line (after removing `quiet splash'):

console=tty0 console=ttyS0,115200n8

Run these commands:

cat >>/etc/inittab <<END
 
# Serial console.
S0:2345:respawn:/sbin/agetty -h -L ttyS0 19200 vt100
END

useradd buildbot

# Password is disabled by default in Centos5.

usermod -a -G wheel buildbot
visudo

# Uncomment the line "%wheel        ALL=(ALL)       NOPASSWD: ALL"
# Comment out this line:
# Defaults    requiretty

# Put in public ssh key for own account and host buildbot account.
# Note that Centos5 seems to require .ssh/authorized_keys chmod go-rwx.

su - buildbot
mkdir .ssh
chmod go-rwx .ssh
cat >.ssh/authorized_keys
chmod go-rwx .ssh/authorized_keys

Image for rpm build

qemu-img create -b vm-centos5-amd64-serial.qcow2 -f qcow2 vm-centos5-amd64-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-centos5-amd64-build.qcow2 -cdrom /kvm/CentOS-5.3-x86_64-bin-DVD.iso -redir 'tcp:2237::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

Install compilers etc:

sudo yum groupinstall "Development Tools"
sudo yum install gperf readline-devel ncurses-devel libaio-devel openssl-devel zlib-devel perl perl\(DBI\)

Download 5.0 rpm for shared-compat:

sudo mkdir -p /srv/shared/yum/CentOS/5/x86_64/RPMS/
cd /srv/shared/yum/CentOS/5/x86_64/RPMS/
sudo wget http://mirror.ourdelta.org/yum/CentOS/5/x86_64/RPMS/MySQL-OurDelta-shared-5.0.87.d10-65.el5.x86_64.rpm

Image for install/test

qemu-img create -b vm-centos5-amd64-serial.qcow2 -f qcow2 vm-centos5-amd64-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-centos5-amd64-install.qcow2 -cdrom /kvm/CentOS-5.3-x86_64-bin-DVD.iso -redir 'tcp:2237::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

Install extra dependencies:

sudo yum install perl perl\(DBI\)

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - CentOS 5 i386

Base install

cd /kvm
qemu-img create -f qcow2 vms/vm-centos5-i386-base.qcow2 8G
# ISO (dvd) install:
kvm -m 2047 -hda /kvm/vms/vm-centos5-i386-base.qcow2 -cdrom CentOS-5.3-i386-bin-DVD.iso -redir 'tcp:2225::22' -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user

Configure for serial console

(cd vms && qemu-img create -b vm-centos5-i386-base.qcow2 -f qcow2 vm-centos5-i386-serial.qcow2)
kvm -m 2047 -hda /kvm/vms/vm-centos5-i386-serial.qcow2 -cdrom CentOS-5.3-i386-bin-DVD.iso -redir 'tcp:2225::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

Add to /boot/grub/menu.lst:

serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial console

also add in menu.lst to kernel line (after removing `quiet splash'):

console=tty0 console=ttyS0,115200n8

Add login prompt on serial console:

cat >>/etc/inittab <<END

# Serial console.
S0:2345:respawn:/sbin/agetty -h -L ttyS0 19200 vt100
END

Create account.

useradd buildbot
# Password is disabled by default in Centos5.
usermod -a -G wheel buildbot
visudo
# Uncomment the line "%wheel        ALL=(ALL)       NOPASSWD: ALL"
# Comment out this line:
# Defaults    requiretty

# Put in public ssh key for own account and host buildbot account.
# Note that Centos5 seems to require .ssh/authorized_keys chmod go-rwx.
su - buildbot
mkdir .ssh
chmod go-rwx .ssh
cat >.ssh/authorized_keys
# Paste the id_dsa.pub key, see above.
chmod go-rwx .ssh/authorized_keys

Image for rpm build

qemu-img create -b vm-centos5-i386-serial.qcow2 -f qcow2 vm-centos5-i386-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-centos5-i386-build.qcow2 -cdrom /kvm/CentOS-5.3-i386-bin-DVD.iso -redir 'tcp:2225::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

Install compilers etc:

sudo yum groupinstall "Development Tools"
sudo yum install gperf readline-devel ncurses-devel zlib-devel libaio-devel openssl-devel perl perl\(DBI\)

Download 5.0 rpm for shared-compat:

sudo mkdir -p /srv/shared/yum/CentOS/5/i386/RPMS/
cd /srv/shared/yum/CentOS/5/i386/RPMS/
sudo wget http://mirror.ourdelta.org/yum/CentOS/5/i386/RPMS/MySQL-OurDelta-shared-5.0.87.d10-65.el5.i386.rpm

Image for install/test

qemu-img create -b vm-centos5-i386-serial.qcow2 -f qcow2 vm-centos5-i386-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-centos5-i386-install.qcow2 -cdrom /kvm/CentOS-5.3-i386-bin-DVD.iso -redir 'tcp:2225::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

Install extra dependencies:

sudo yum install perl perl\(DBI\)

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - CentOS 6.2

Base install

qemu-img create -f qcow2 /kvm/vms/vm-centos6-amd64-serial.qcow2 8G
qemu-img create -f qcow2 /kvm/vms/vm-centos6-i386-serial.qcow2 8G

Start each VM booting from the server install iso one at a time and perform the following install steps:

kvm -m 1024 -hda /kvm/vms/vm-centos6-amd64-serial.qcow2 -cdrom /kvm/iso/centos/CentOS-6.2-x86_64-bin-DVD1.iso -redir tcp:22255::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 1024 -hda /kvm/vms/vm-centos6-i386-serial.qcow2 -cdrom /kvm/iso/centos/CentOS-6.2-i386-bin-DVD1.iso -redir tcp:22256::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user

Once running you can connect to the VNC server from your local host with:

vncviewer -via ${remote-host} localhost

Replace ${remote-host} with the host the vm is running on.

Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. The CentOS installer has just resized the vnc screen. Simply reconnect.

Install, picking default options mostly, with the following notes:

The Installer will throw up a "Storage Device Warning", choose "Yes, discard any data"
Set the hostname to centos6-amd64 (or centos6-i386)
Click the "Configure Network" button on the Hostname screen.
- Edit System eth0 to "connect automatically"
- Apply and then close the "Network Connections" window
Set Timezone to Europe/Helsinki (keep "System clock uses UTC" checked)
When partitioning disks, choose "Use All Space"
- do not check the "Encrypt system" checkbox
- do check the "Review and modify partitioning layout" checkbox
- Delete the LVM stuff and leaving the sda1 partition alone, repartition the physical volume as follows

Device

Size(MB)

Mount Point

Type

Format

sda2

5672

ext4

yes

sda3

(max allowable)

(n/a)

swap

yes

Minimal install
Customize Later

When the install is finished, you will be prompted to reboot. Go ahead and do so, but it will fail. Kill the VM (after the reboot fails) and start it up again:

kvm -m 1024 -hda /kvm/vms/vm-centos6-amd64-serial.qcow2 -redir tcp:22255::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user

kvm -m 1024 -hda /kvm/vms/vm-centos6-i386-serial.qcow2 -redir tcp:22256::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user

You may connect via VNC as before, but ssh is probably preferred. Login as root.

Now that the VM is installed, it's time to configure it.

ssh -p 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhost
ssh -p 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhost

After logging in as root, create a local user

adduser ${username}
usermod -a -G wheel ${username}
passwd ${username}

Enable password-less sudo and serial console:

visudo
# Uncomment the line "%wheel        ALL=(ALL)       NOPASSWD: ALL"
# Comment out this line:
# Defaults    requiretty

Still logged in as root, add to /boot/grub/menu.lst:

serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial console

also add in menu.lst to kernel line (after removing 'quiet'):

console=tty0 console=ttyS0,115200n8

Add login prompt on serial console:

cat >>/etc/inittab <<END

# Serial console.
S0:2345:respawn:/sbin/agetty -h -L ttyS0 19200 vt100
END

Logout as root, and then, from the VM host server:

Install proper ssh:

ssh -t -p 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "sudo yum install openssh-server openssh-clients"
ssh -t -p 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "sudo yum install openssh-server openssh-clients"

Create a .ssh folder:

ssh -t -p 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"
ssh -t -p 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"

Copy over the authorized keys file:

scp -P 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/
scp -P 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/

Set permissions on the .ssh folder correctly:

ssh -t -p 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"
ssh -t -p 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"

Create the buildbot user:

ssh -p 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
ssh -p 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'

Upload the ttyS0.conf file and put it where it goes:

scp -P 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0.conf buildbot@localhost:
scp -P 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0.conf buildbot@localhost:

ssh -p 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf;'
ssh -p 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf;'

Update the VM:

ssh -p 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

Once logged in:

sudo yum update

After updating, shut down the VM:

sudo shutdown -h now

VMs for building .rpms

for i in '/kvm/vms/vm-centos6-amd64-serial.qcow2 22255 qemu64' '/kvm/vms/vm-centos6-i386-serial.qcow2 22256 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
    "sudo yum -y groupinstall 'Development Tools'" \
    "sudo yum -y install wget tree gperf readline-devel ncurses-devel zlib-devel pam-devel libaio-devel openssl-devel libxml2 libxml2-devel unixODBC-devel bzr perl perl\(DBI\)" \
    "sudo yum -y remove systemtap-sdt-dev" \
    "bzr co --lightweight lp:mariadb-native-client" \
    "sudo mkdir -vp /usr/src/redhat/SOURCES /usr/src/redhat/SPECS /usr/src/redhat/RPMS /usr/src/redhat/SRPMS" \
    "wget http://www.cmake.org/files/v2.8/cmake-2.8.8.tar.gz;tar -zxvf cmake-2.8.8.tar.gz;cd cmake-2.8.8;./configure;make;sudo make install"; \
done

Also:

Installing the Boost library needed for the OQGraph storage engine

VMs for install testing.

MariaDB.local.repo points at a local directory on the VM. MariaDB.repo points at the real MariaDB YUM repository.

for i in '/kvm/vms/vm-centos6-amd64-serial.qcow2 22255 qemu64' '/kvm/vms/vm-centos6-i386-serial.qcow2 22256 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
    "sudo yum -y update" \
    "sudo yum -y install patch libaio perl perl-Time-HiRes perl-DBI libtool-ltdl unixODBC" \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
    "sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/"; \
done

VMs for MySQL upgrade testing

for i in '/kvm/vms/vm-centos6-amd64-serial.qcow2 22255 qemu64' '/kvm/vms/vm-centos6-i386-serial.qcow2 22256 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
    "sudo yum -y update" \
    'sudo yum -y install mysql-server libtool-ltdl unixODBC' \
    'sudo /etc/init.d/mysqld start' \
    'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
    "sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/"; \
done

The MariaDB upgrade testing VMs were not built. There is currently an error with installing MariaDB from the YUM repo.

VMs for MariaDB upgrade testing

for i in '/kvm/vms/vm-centos6-amd64-serial.qcow2 22255 qemu64' '/kvm/vms/vm-centos6-i386-serial.qcow2 22256 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
    'sudo yum -y update' \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.repo buildbot@localhost:/tmp/" \
    'sudo rpm --verbose --import http://downloads.mariadb.org/repo/RPM-GPG-KEY-MariaDB' \
    'sudo mv -vi /tmp/MariaDB.repo /etc/yum.repos.d/' \
    'sudo yum -y remove mysql-libs' \
    'sudo yum -y install MariaDB-server MariaDB-client MariaDB-test' \
    'sudo yum -y install cronie cronie-anacron crontabs.noarch postfix' \
    'sudo /etc/init.d/mysqld start' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
    'sudo rm -v /etc/yum.repos.d/MariaDB.repo' \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
    'sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/'; \
done

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Fedora 16

Base install

qemu-img create -f qcow2 /kvm/vms/vm-fedora16-i386-serial.qcow2 10G
qemu-img create -f qcow2 /kvm/vms/vm-fedora16-amd64-serial.qcow2 10G

Start each VM booting from the server install iso one at a time and perform the following install steps:

kvm -m 1024 -hda /kvm/vms/vm-fedora16-i386-serial.qcow2 -cdrom /kvm/iso/fedora/Fedora-16-i386-DVD.iso -redir tcp:2263::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
kvm -m 1024 -hda /kvm/vms/vm-fedora16-amd64-serial.qcow2 -cdrom /kvm/iso/fedora/Fedora-16-x86_64-DVD.iso -redir tcp:2264::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Once running you can connect to the VNC server from your local host with:

vncviewer -via ${remote-host} localhost

Replace ${remote-host} with the host the vm is running on.

Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. The Fedora installer has just resized the vnc screen. Simply reconnect.

Install, picking default options mostly, with the following notes:

The Installer will throw up a "Storage Device Warning", choose "Yes, discard any data"
Set the hostname to fedora16-amd64 (or fedora16-i386)
Click the "Configure Network" button on the Hostname screen.
- Edit System eth0 to "connect automatically"
- Apply and then close the "Network Connections" window
When partitioning disks, choose "Use All Space"
- uncheck the "Use LVM" checkbox
- do not check the "Encrypt system" checkbox
Minimal install
Customize Later

When the install is finished, you will be prompted to reboot. Go ahead and do so, but it will fail. Kill the VM (after the reboot fails) and start it up again:

kvm -m 1024 -hda /kvm/vms/vm-fedora16-i386-serial.qcow2 -redir tcp:2263::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
kvm -m 1024 -hda /kvm/vms/vm-fedora16-amd64-serial.qcow2 -redir tcp:2264::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Until the extra user is installed you must connect via VNC as before. SSH is preferred, so that's what we'll do first. Login as root.

ssh -p 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhost
ssh -p 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhost

After logging in as root, install proper ssh and then create a local user:

/sbin/chkconfig --level 35 network on
ifup eth0
yum install openssh-server openssh-clients
adduser ${username}
usermod -a -G wheel ${username}
passwd ${username}

Enable password-less sudo and serial console:

visudo
# Uncomment the line "%wheel        ALL=(ALL)       NOPASSWD: ALL"
# Comment out this line:
# Defaults    requiretty

Still logged in as root, add to /boot/grub/menu.lst:

Editing /boot/grub/menu.lst:

sudo vi /etc/default/grub

# Add/edit these entries:
    GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
    GRUB_TERMINAL="serial"
    GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

grub2-mkconfig -o /boot/grub2/grub.cfg

Logout as root, and then, from the VM host server:

Create a .ssh folder:

ssh -t -p 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"
ssh -t -p 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"

Copy over the authorized keys file:

scp -P 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/

Set permissions on the .ssh folder correctly:

ssh -t -p 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"
ssh -t -p 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"

Create the buildbot user:

ssh -p 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
ssh -p 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'

su to the local buildbot user and ssh to the vm to put the key in known_hosts:

For i386:

sudo su - buildbot
ssh -p 2263 buildbot@localhost
# exit, then exit again

For amd64:

sudo su - buildbot
ssh -p 2264 buildbot@localhost
# exit, then exit again

Upload the ttyS0 file and put it where it goes:

scp -P 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:
scp -P 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:

ssh -p 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mv -vi ttyS0 /etc/event.d/;'
ssh -p 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mv -vi ttyS0 /etc/event.d/;'

Update the VM:

ssh -p 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

Once logged in:

sudo yum update

After updating, shut down the VM:

sudo shutdown -h now

VMs for building .rpms

for i in '/kvm/vms/vm-fedora16-amd64-serial.qcow2 2264 qemu64' '/kvm/vms/vm-fedora16-i386-serial.qcow2 2263 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
    "sudo yum -y groupinstall 'Development Tools'" \
    "sudo yum -y install wget tree gperf readline-devel ncurses-devel zlib-devel pam-devel libaio-devel openssl-devel perl perl\(DBI\)" \
    "sudo yum -y remove systemtap-sdt-dev" \
    "sudo mkdir -vp /usr/src/redhat/SOURCES /usr/src/redhat/SPECS /usr/src/redhat/RPMS /usr/src/redhat/SRPMS" \
    "wget http://www.cmake.org/files/v2.8/cmake-2.8.8.tar.gz;tar -zxvf cmake-2.8.8.tar.gz;cd cmake-2.8.8;./configure;make;sudo make install"; \
done

Also:

Installing the Boost library needed for the OQGraph storage engine

VMs for install testing.

MariaDB.local.repo points at a local directory on the VM. MariaDB.repo points at the real MariaDB YUM repository.

for i in '/kvm/vms/vm-fedora16-amd64-serial.qcow2 2264 qemu64' '/kvm/vms/vm-fedora16-i386-serial.qcow2 2263 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
    "sudo yum -y update" \
    "sudo yum -y install libaio perl perl-Time-HiRes perl-DBI" \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
    "sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/"; \
done

VMs for MySQL upgrade testing

for i in '/kvm/vms/vm-fedora16-amd64-serial.qcow2 2264 qemu64' '/kvm/vms/vm-fedora16-i386-serial.qcow2 2263 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
    "sudo yum -y update" \
    'sudo yum -y install mysql-server' \
    'sudo systemctl enable mysqld.service' \
    'sudo systemctl start mysqld.service' \
    'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
    "sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/"; \
done

The MariaDB upgrade testing VMs were not built. Once we have MariaDB Fedora 16 RPMs, then I will attempt building this VM. For now, the placeholder text below is copied from the Buildbot Setup for Virtual Machines - CentOS 6.2 page.

VMs for MariaDB upgrade testing

for i in '/kvm/vms/vm-fedora16-amd64-serial.qcow2 2264 qemu64' '/kvm/vms/vm-fedora16-i386-serial.qcow2 2263 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
    'sudo yum -y update' \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.repo buildbot@localhost:/tmp/" \
    'sudo rpm --verbose --import http://downloads.mariadb.org/repo/RPM-GPG-KEY-MariaDB' \
    'sudo mv -vi /tmp/MariaDB.repo /etc/yum.repos.d/' \
    'sudo yum -y remove mysql-libs' \
    'sudo yum -y install MariaDB-server MariaDB-client MariaDB-test' \
    'sudo yum -y install cronie cronie-anacron crontabs.noarch postfix' \
    'sudo /etc/init.d/mysqld start' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
    'sudo rm -v /etc/yum.repos.d/MariaDB.repo' \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
    'sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/'; \
done

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Fedora 17

Base install

qemu-img create -f qcow2 /kvm/vms/vm-fedora17-i386-serial.qcow2 10G
qemu-img create -f qcow2 /kvm/vms/vm-fedora17-amd64-serial.qcow2 10G

Start each VM booting from the server install iso one at a time and perform the following install steps:

kvm -m 1024 -hda /kvm/vms/vm-fedora17-i386-serial.qcow2 -cdrom /kvm/iso/fedora/Fedora-17-i386-DVD.iso -redir tcp:2265::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
kvm -m 1024 -hda /kvm/vms/vm-fedora17-amd64-serial.qcow2 -cdrom /kvm/iso/fedora/Fedora-17-x86_64-DVD.iso -redir tcp:2266::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Once running you can connect to the VNC server from your local host with:

vncviewer -via ${remote-host} localhost

Replace ${remote-host} with the host the vm is running on.

Install, picking default options mostly, with the following notes:

The Installer will throw up a "Storage Device Warning", choose "Yes, discard any data"
Set the hostname to fedora17-amd64 (or fedora17-i386)
Click the "Configure Network" button on the Hostname screen.
- Edit System eth0 to "connect automatically"
- Apply and then close the "Network Connections" window
When partitioning disks, choose "Use All Space"
- uncheck the "Use LVM" checkbox
- do not check the "Encrypt system" checkbox
Minimal install
Customize Later

When the install is finished, you will be prompted to reboot. Go ahead and do so, but it will fail. Kill the VM (after the reboot fails) and start it up again:

kvm -m 1024 -hda /kvm/vms/vm-fedora17-i386-serial.qcow2 -redir tcp:2265::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 1024 -hda /kvm/vms/vm-fedora17-amd64-serial.qcow2 -redir tcp:2266::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Until the extra user is installed you must connect via VNC as before. SSH is preferred, so that's what we'll do first. Login as root.

ssh -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhost
ssh -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhost

After logging in as root, install proper ssh and then create a local user:

/sbin/chkconfig --level 35 network on
ifup eth0
yum install openssh-server openssh-clients
adduser ${username}
usermod -a -G wheel ${username}
passwd ${username}

Enable password-less sudo and serial console:

visudo
# Uncomment the line "%wheel        ALL=(ALL)       NOPASSWD: ALL"
# Comment out this line:
# Defaults    requiretty

Still logged in as root, add to /boot/grub/menu.lst:

Editing /boot/grub/menu.lst:

sudo vi /etc/default/grub

# Add/edit these entries:
    GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
    GRUB_TERMINAL="serial"
    GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

grub2-mkconfig -o /boot/grub2/grub.cfg

Logout as root, and then, from the VM host server:

Create a .ssh folder:

ssh -t -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"
ssh -t -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"

Copy over the authorized keys file:

scp -P 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/

Set permissions on the .ssh folder correctly:

ssh -t -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"
ssh -t -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"

Create the buildbot user:

ssh -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
ssh -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'

su to the local buildbot user and ssh to the vm to put the key in known_hosts:

For i386:

sudo su - buildbot
ssh -p 2265 buildbot@localhost
# exit, then exit again

For amd64:

sudo su - buildbot
ssh -p 2266 buildbot@localhost
# exit, then exit again

Upload the ttyS0 file and put it where it goes:

scp -P 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:
scp -P 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:

ssh -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'
ssh -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'

Update the VM:

ssh -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

Once logged in:

sudo yum update

After updating, shut down the VM:

sudo shutdown -h now

VMs for building .rpms

On the VM host server:

qemu-img create -b /kvm/vms/vm-fedora17-i386-serial.qcow2 -f qcow2 /kvm/vms/vm-fedora17-i386-build.qcow2
qemu-img create -b /kvm/vms/vm-fedora17-amd64-serial.qcow2 -f qcow2 /kvm/vms/vm-fedora17-amd64-build.qcow2

Boot the VMs with:

kvm -m 1024 -hda /kvm/vms/vm-fedora17-i386-build.qcow2 -redir tcp:2265::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
kvm -m 1024 -hda /kvm/vms/vm-fedora17-amd64-build.qcow2 -redir tcp:2266::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

Copy over the boost tar file:

scp -P 2265 -i /kvm/vms/ssh-keys/id_dsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null /kvm/boost_1_49_0.tar.gz buildbot@localhost:/dev/shm/
scp -P 2266 -i /kvm/vms/ssh-keys/id_dsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null /kvm/boost_1_49_0.tar.gz buildbot@localhost:/dev/shm/

ssh -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

Once logged in:

sudo yum -y groupinstall 'Development Tools'
sudo yum -y install tar wget tree gperf readline-devel ncurses-devel zlib-devel pam-devel libaio-devel openssl-devel perl perl\(DBI\)
sudo yum -y remove systemtap-sdt-dev
sudo mkdir -vp /usr/src/redhat/SOURCES /usr/src/redhat/SPECS /usr/src/redhat/RPMS /usr/src/redhat/SRPMS

# cmake
wget http://www.cmake.org/files/v2.8/cmake-2.8.8.tar.gz;tar -zxvf cmake-2.8.8.tar.gz;cd cmake-2.8.8;./configure;make;sudo make install

# boost
cd /usr/local/src;sudo tar zxf /dev/shm/boost_1_49_0.tar.gz;cd /usr/local/include/;sudo ln -s ../src/boost_1_49_0/boost .

# shutdown the VM
sudo shutdown -h now

VMs for install testing.

On the VM host server:

qemu-img create -b /kvm/vms/vm-fedora17-i386-serial.qcow2 -f qcow2 /kvm/vms/vm-fedora17-i386-install.qcow2
qemu-img create -b /kvm/vms/vm-fedora17-amd64-serial.qcow2 -f qcow2 /kvm/vms/vm-fedora17-amd64-install.qcow2

Boot the VMs with:

kvm -m 1024 -hda /kvm/vms/vm-fedora17-i386-install.qcow2 -redir tcp:2265::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
kvm -m 1024 -hda /kvm/vms/vm-fedora17-amd64-install.qcow2 -redir tcp:2266::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

ssh -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

Once logged in:

sudo yum -y update
sudo yum -y install patch tar libaio perl perl-Time-HiRes perl-DBI

# shutdown the VM
sudo shutdown -h now

VMs for MySQL upgrade testing

On the VM host server:

qemu-img create -b /kvm/vms/vm-fedora17-i386-serial.qcow2 -f qcow2 /kvm/vms/vm-fedora17-i386-upgrade.qcow2
qemu-img create -b /kvm/vms/vm-fedora17-amd64-serial.qcow2 -f qcow2 /kvm/vms/vm-fedora17-amd64-upgrade.qcow2

Boot the VMs with:

kvm -m 1024 -hda /kvm/vms/vm-fedora17-i386-upgrade.qcow2 -redir tcp:2265::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
kvm -m 1024 -hda /kvm/vms/vm-fedora17-amd64-upgrade.qcow2 -redir tcp:2266::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

ssh -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

Once logged in:

sudo yum -y update
sudo yum -y install tar mysql-server
sudo systemctl enable mysqld.service
sudo systemctl start mysqld.service
mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"

# shutdown the VM
sudo shutdown -h now

The MariaDB upgrade testing VMs were not built. Once we have MariaDB Fedora 17 RPMs, then I will attempt building this VM. For now, the placeholder text below is copied from the Buildbot Setup for Virtual Machines - CentOS 6.2 page.

VMs for MariaDB upgrade testing

for i in '/kvm/vms/vm-fedora17-amd64-serial.qcow2 2266 qemu64' '/kvm/vms/vm-fedora17-i386-serial.qcow2 2265 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
    'sudo yum -y update' \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.repo buildbot@localhost:/tmp/" \
    'sudo rpm --verbose --import http://downloads.mariadb.org/repo/RPM-GPG-KEY-MariaDB' \
    'sudo mv -vi /tmp/MariaDB.repo /etc/yum.repos.d/' \
    'sudo yum -y remove mysql-libs' \
    'sudo yum -y install MariaDB-server MariaDB-client MariaDB-test' \
    'sudo yum -y install cronie cronie-anacron crontabs.noarch postfix' \
    'sudo /etc/init.d/mysqld start' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
    'sudo rm -v /etc/yum.repos.d/MariaDB.repo' \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
    'sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/'; \
done

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Fedora 18

Base install

qemu-img create -f qcow2 /kvm/vms/vm-fedora18-i386-serial.qcow2 10G
qemu-img create -f qcow2 /kvm/vms/vm-fedora18-amd64-serial.qcow2 10G

Start each VM booting from the server install iso one at a time and perform the following install steps:

kvm -m 2048 -hda /kvm/vms/vm-fedora18-i386-serial.qcow2 -cdrom /kvm/iso/fedora/Fedora-18-i386-DVD.iso -redir tcp:2277::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 2048 -hda /kvm/vms/vm-fedora18-amd64-serial.qcow2 -cdrom /kvm/iso/fedora/Fedora-18-x86_64-DVD.iso -redir tcp:2278::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Once running you can connect to the VNC server from your local host with:

vncviewer -via ${remote-host} localhost

Replace ${remote-host} with the host the vm is running on.

Install, picking default options mostly, with the following notes:

Under "Network Configuration" set the hostnames to fedora18-amd64 and fedora18-i386
Change "Software Selection" to "Minimal Install" (default is "Gnome Desktop")
For "Installation Destination" select the disk then click continue.
do not check the encryption checkbox
On "Installation Options" screen, expand the "Partition scheme configuration" box and select a "Partition type" of "Standard Partition"
While installing, set the root password

When the install is finished, you will be prompted to reboot. Go ahead and do so, but it will fail. Kill the VM (after the reboot fails) and start it up again:

kvm -m 2048 -hda /kvm/vms/vm-fedora18-i386-serial.qcow2 -redir tcp:2277::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 2048 -hda /kvm/vms/vm-fedora18-amd64-serial.qcow2 -redir tcp:2278::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Until a non-root user is installed you must connect as root. SSH is preferred, so that's what we'll do first. Login as root.

ssh -p 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhost
ssh -p 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhost

After logging in as root, install proper ssh and then create a local user:

adduser ${username}
usermod -a -G wheel ${username}
passwd ${username}

Enable password-less sudo and serial console:

visudo
# Uncomment the line "%wheel        ALL=(ALL)       NOPASSWD: ALL"
# Comment out this line:
# Defaults    requiretty

Still logged in as root, add to /boot/grub/menu.lst:

Editing /boot/grub/menu.lst:

sudo vi /etc/default/grub

# Add/edit these entries:
    GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
    GRUB_TERMINAL="serial"
    GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

grub2-mkconfig -o /boot/grub2/grub.cfg

Logout as root, and then, from the VM host server:

Create a .ssh folder:

ssh -t -p 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"
ssh -t -p 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"

Copy over the authorized keys file:

scp -P 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/

Set permissions on the .ssh folder correctly:

ssh -t -p 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"
ssh -t -p 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"

Create the buildbot user:

ssh -p 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
ssh -p 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'

su to the local buildbot user and ssh to the vm to put the key in known_hosts:

For i386:

sudo su - buildbot
ssh -p 2277 buildbot@localhost
# exit, then exit again

For amd64:

sudo su - buildbot
ssh -p 2278 buildbot@localhost
# exit, then exit again

Upload the ttyS0 file and put it where it goes:

scp -P 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:
scp -P 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:

ssh -p 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'
ssh -p 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'

Update the VM:

ssh -p 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

Once logged in:

sudo yum update

After updating, shut down the VM:

sudo shutdown -h now

VMs for building .rpms

for i in '/kvm/vms/vm-fedora18-i386-serial.qcow2 2277 qemu64' '/kvm/vms/vm-fedora18-amd64-serial.qcow2 2278 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
  "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/boost_1_49_0.tar.gz buildbot@localhost:/dev/shm/" \
  "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/thrift-0.9.0.tar.gz buildbot@localhost:/dev/shm/" \
  "sudo yum -y groupinstall 'Development Tools'" \
  "sudo yum -y install cmake tar wget tree gperf readline-devel ncurses-devel zlib-devel pam-devel libaio-devel openssl-devel libxml2 libxml2-devel unixODBC-devel bzr perl perl\(DBI\)" \
  "sudo yum -y remove systemtap-sdt-dev" \
  "bzr co --lightweight lp:mariadb-native-client" \
  "sudo mkdir -vp /usr/src/redhat/SOURCES /usr/src/redhat/SPECS /usr/src/redhat/RPMS /usr/src/redhat/SRPMS" \
  "cd /usr/local/src;sudo tar zxf /dev/shm/thrift-0.9.0.tar.gz;pwd;ls" \
  "cd /usr/local/src/thrift-0.9.0;echo;pwd;sudo ./configure --prefix=/usr --enable-shared=no --enable-static=yes CXXFLAGS=-fPIC CFLAGS=-fPIC && echo && echo 'now making' && echo && sleep 3 && sudo make && echo && echo 'now installing' && echo && sleep 3 && sudo make install" \
  "cd /usr/local/src;sudo tar zxf /dev/shm/boost_1_49_0.tar.gz;cd /usr/local/include/;sudo ln -vs ../src/boost_1_49_0/boost ." ; \
done

VMs for install testing.

for i in '/kvm/vms/vm-fedora18-i386-serial.qcow2 2277 qemu64' '/kvm/vms/vm-fedora18-amd64-serial.qcow2 2278 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
    "sudo yum -y update" \
    "sudo yum -y install patch tar libaio perl perl-Time-HiRes perl-DBI" ; \
done

VMs for MySQL upgrade testing

for i in '/kvm/vms/vm-fedora18-i386-serial.qcow2 2277 qemu64' '/kvm/vms/vm-fedora18-amd64-serial.qcow2 2278 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
    "sudo yum -y update" \
    "sudo yum -y install patch tar mysql-server libtool-ltdl unixODBC" \
    "sudo systemctl enable mysqld.service" \
    "sudo systemctl start mysqld.service" \
    'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ; \
done

VMs for MariaDB upgrade testing

for i in '/kvm/vms/vm-fedora18-amd64-serial.qcow2 2278 qemu64' '/kvm/vms/vm-fedora18-i386-serial.qcow2 2277 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
    'sudo yum -y update' \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB-${2}.repo buildbot@localhost:/tmp/MariaDB.repo" \
    'sudo rpm --verbose --import https://yum.mariadb.org/RPM-GPG-KEY-MariaDB' \
    'sudo mv -vi /tmp/MariaDB.repo /etc/yum.repos.d/' \
    'sudo yum -y remove mysql-libs' \
    'sudo yum -y install MariaDB-server MariaDB-client MariaDB-test libtool-ltdl unixODBC' \
    'sudo yum -y install cronie cronie-anacron crontabs.noarch postfix patch tar' \
    'sudo /etc/init.d/mysqld start' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
    'sudo rm -v /etc/yum.repos.d/MariaDB.repo' \
    'sudo yum -y update' \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
    'sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/'; \
done

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Fedora 19

Base install

qemu-img create -f qcow2 /kvm/vms/vm-fedora19-i386-serial.qcow2 20G
qemu-img create -f qcow2 /kvm/vms/vm-fedora19-amd64-serial.qcow2 20G

Start each VM booting from the server install iso one at a time and perform the following install steps:

kvm -m 2048 -hda /kvm/vms/vm-fedora19-i386-serial.qcow2 -cdrom /ds413/iso/fedora/Fedora-19-i386-DVD.iso -redir tcp:2289::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 2048 -hda /kvm/vms/vm-fedora19-amd64-serial.qcow2 -cdrom /ds413/iso/fedora/Fedora-19-x86_64-DVD.iso -redir tcp:2290::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Once running you can connect to the VNC server from your local host with:

vncviewer -via ${remote-host} localhost

Replace ${remote-host} with the host the vm is running on.

Install, picking default options mostly, with the following notes:

Under "Network Configuration" set the hostnames to fedora19-amd64 and fedora19-i386
Change "Software Selection" to "Minimal Install" (default is "Gnome Desktop")
For "Installation Destination" select the disk then click continue.
do not check the encryption checkbox
On "Installation Options" screen, expand the "Partition scheme configuration" box and select a "Partition type" of "Standard Partition"
While installing, set the root password and an initial user

When the install is finished, you will be prompted to reboot. Go ahead and do so, but it will fail. Kill the VM (after the reboot fails) and start it up again:

kvm -m 2048 -hda /kvm/vms/vm-fedora19-i386-serial.qcow2 -redir tcp:2289::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 2048 -hda /kvm/vms/vm-fedora19-amd64-serial.qcow2 -redir tcp:2290::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user

ssh -p 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost
ssh -p 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost

After logging in, add the initial user to the "wheel" group.

usermod -a -G wheel ${username}

Enable password-less sudo for the "wheel" group and serial console sudo:

visudo
# Uncomment the line "%wheel        ALL=(ALL)       NOPASSWD: ALL"
# Comment out this line:
# Defaults    requiretty

Edit /boot/grub/menu.lst:

sudo vi /etc/default/grub

# Add/edit these entries:
    GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
    GRUB_TERMINAL="serial"
    GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

grub2-mkconfig -o /boot/grub2/grub.cfg

Logout as root, and then, from the VM host server:

Create a .ssh folder:

ssh -t -p 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"
ssh -t -p 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"

Copy over the authorized keys file:

scp -P 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/

Set permissions on the .ssh folder correctly:

ssh -t -p 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"
ssh -t -p 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"

Create the buildbot user:

ssh -p 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
ssh -p 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'

su to the local buildbot user and ssh to the vm to put the key in known_hosts:

For i386:

sudo su - buildbot
ssh -p 2289 buildbot@localhost
# exit, then exit again

For amd64:

sudo su - buildbot
ssh -p 2290 buildbot@localhost
# exit, then exit again

Upload the ttyS0 file and put it where it goes:

scp -P 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:
scp -P 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:

ssh -p 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'
ssh -p 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'

Update the VM:

ssh -p 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

Once logged in:

sudo yum update

Change selinux policy to permissive:

sudo vi /etc/selinux/config 
# change:
SELINUX=enforcing
# to:
SELINUX=permissive

After updating, shut down the VM:

sudo shutdown -h now

VMs for building .rpms

for i in '/kvm/vms/vm-fedora19-i386-serial.qcow2 2289 qemu64' '/kvm/vms/vm-fedora19-amd64-serial.qcow2 2290 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
  "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/thrift-0.9.0.tar.gz buildbot@localhost:/dev/shm/" \
  "sudo yum -y groupinstall 'Development Tools'" \
  "sudo yum -y install yum-utils" \
  "sudo yum-builddep -y mariadb" \
  "sudo yum -y install automake libtool flex pkgconfig gcc-c++ libevent-devel python-devel ruby-devel rpm-build" \
  "sudo yum -y install cmake tar wget tree gperf readline-devel ncurses-devel zlib-devel pam-devel libaio-devel openssl-devel" \
  "sudo yum -y install libxml2 libxml2-devel bison bison-devel boost-devel unixODBC-devel bzr perl perl\(DBI\)" \
  "sudo yum -y remove systemtap-sdt-dev" \
  "bzr co --lightweight lp:mariadb-native-client" \
  "sudo mkdir -vp /usr/src/redhat/SOURCES /usr/src/redhat/SPECS /usr/src/redhat/RPMS /usr/src/redhat/SRPMS" \
  "cd /usr/local/src;sudo tar zxf /dev/shm/thrift-0.9.0.tar.gz;pwd;ls" \
  "cd /usr/local/src/thrift-0.9.0;echo;pwd;sudo ./configure --prefix=/usr --enable-shared=no --enable-static=yes CXXFLAGS=-fPIC CFLAGS=-fPIC && echo && echo 'now making' && echo && sleep 3 && sudo make && echo && echo 'now installing' && echo && sleep 3 && sudo make install" ; \
done

VMs for install testing.

for i in '/kvm/vms/vm-fedora19-i386-serial.qcow2 2289 qemu64' '/kvm/vms/vm-fedora19-amd64-serial.qcow2 2290 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
    "sudo yum -y update" \
    "sudo yum -y install patch tar libaio perl perl-Time-HiRes perl-DBI unixODBC" ; \
done

VMs for MySQL upgrade testing

for i in '/kvm/vms/vm-fedora19-i386-serial.qcow2 2289 qemu64' '/kvm/vms/vm-fedora19-amd64-serial.qcow2 2290 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
    "sudo yum -y update" \
    "sudo yum -y install patch tar mysql-server libtool-ltdl unixODBC" \
    "sudo systemctl enable mysqld.service" \
    "sudo systemctl start mysqld.service" \
    'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ; \
done

VMs for MariaDB upgrade testing

for i in '/kvm/vms/vm-fedora19-amd64-serial.qcow2 2290 qemu64' '/kvm/vms/vm-fedora19-i386-serial.qcow2 2289 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
    'sudo yum -y update' \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB-${2}.repo buildbot@localhost:/tmp/MariaDB.repo" \
    'sudo rpm --verbose --import https://yum.mariadb.org/RPM-GPG-KEY-MariaDB' \
    'sudo mv -vi /tmp/MariaDB.repo /etc/yum.repos.d/' \
    'sudo yum -y remove mysql-libs' \
    'sudo yum -y install MariaDB-server MariaDB-client MariaDB-test libtool-ltdl unixODBC' \
    'sudo yum -y install cronie cronie-anacron crontabs.noarch patch tar' \
    'sudo /etc/init.d/mysql start' \
    'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
    'sudo rm -v /etc/yum.repos.d/MariaDB.repo' \
    'sudo yum -y update' \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
    'sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/'; \
done

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Fedora 20

Base install

qemu-img create -f qcow2 /kvm/vms/vm-fedora20-i386-serial.qcow2 20G
qemu-img create -f qcow2 /kvm/vms/vm-fedora20-amd64-serial.qcow2 20G

Start each VM booting from the server install iso one at a time and perform the following install steps:

kvm -m 2048 -hda /kvm/vms/vm-fedora20-i386-serial.qcow2 -cdrom /ds413/iso/fedora/Fedora-20-i386-DVD.iso -redir tcp:2291::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 2048 -hda /kvm/vms/vm-fedora20-amd64-serial.qcow2 -cdrom /ds413/iso/fedora/Fedora-20-x86_64-DVD.iso -redir tcp:2292::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Once running you can connect to the VNC server from your local host with:

vncviewer -via ${remote-host} localhost

Replace ${remote-host} with the host the vm is running on.

Install, picking default options mostly, with the following notes:

Set the language to English(US)
Set the timezone to Etc/Greenwich Mean Time timezone
Change "Software Selection" to "Minimal Install" (default is "Gnome Desktop")
Under "Network Configuration" set the hostnames to fedora20-amd64 and fedora20-i386
For "Installation Destination" select the disk then click continue.
- On "Installation Options" screen, select the "Partition scheme" drop-down menu and select "Standard Partition". We do not want LVM.
- do not check the encryption checkbox
Select the "Begin installation" button to start the install
While installing, set the root password and an initial user.
- Be sure the initial user is an administrator

When the install is finished, you will be prompted to reboot. Go ahead and do so, but it will fail. Kill the VM after the reboot fails and start it up again with:

kvm -m 2048 -hda /kvm/vms/vm-fedora20-i386-serial.qcow2 -redir tcp:2291::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 2048 -hda /kvm/vms/vm-fedora20-amd64-serial.qcow2 -redir tcp:2292::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user

ssh -p 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost
ssh -p 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost

After logging in, add the initial user to the "wheel" group.

sudo usermod -a -G wheel ${username}

Enable password-less sudo for the "wheel" group and serial console sudo:

sudo visudo
# Uncomment the line "%wheel        ALL=(ALL)       NOPASSWD: ALL"
# Comment out this line:
# Defaults    requiretty

Edit /boot/grub/menu.lst:

sudo vi /etc/default/grub

# Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

sudo grub2-mkconfig -o /boot/grub2/grub.cfg

Logout and then, from the VM host server:

Create a .ssh folder:

ssh -t -p 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"
ssh -t -p 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"

Copy over the authorized keys file:

scp -P 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/

Set permissions on the .ssh folder correctly:

ssh -t -p 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -vR go-rwx .ssh"
ssh -t -p 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -vR go-rwx .ssh"

Create the buildbot user:

ssh -t -p 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -vR go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
ssh -t -p 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -vR go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'

su to the local buildbot user and ssh to the vm to put the key in known_hosts:

For i386:

sudo su - buildbot
ssh -p 2291 buildbot@localhost
# exit, then exit again

For amd64:

sudo su - buildbot
ssh -p 2292 buildbot@localhost
# exit, then exit again

Upload the ttyS0 file and put it where it goes:

scp -P 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:
scp -P 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:

ssh -t -p 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'
ssh -t -p 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'

Update the VM:

ssh -t -p 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -t -p 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

Once logged in:

sudo yum -y update

Change selinux policy to permissive:

sudo sed -i 's/SELINUX=enforcing/SELINUX=permissive/' /etc/selinux/config

After updating, exit, then shut down the VM:

exit

ssh -t -p 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo shutdown -h now;exit'
ssh -t -p 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo shutdown -h now;exit'

VMs for building .rpms

for i in '/kvm/vms/vm-fedora20-i386-serial.qcow2 2291 qemu64' '/kvm/vms/vm-fedora20-amd64-serial.qcow2 2292 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_build_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
  "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/thrift-0.9.0.tar.gz buildbot@localhost:/dev/shm/" \
  "sudo yum -y groupinstall 'Development Tools'" \
  "sudo yum -y install yum-utils" \
  "sudo yum-builddep -y mariadb" \
  "sudo yum -y install automake libtool flex pkgconfig gcc-c++ libevent-devel python-devel ruby-devel rpm-build Judy-devel" \
  "sudo yum -y install cmake tar wget tree gperf readline-devel ncurses-devel zlib-devel pam-devel libaio-devel openssl-devel" \
  "sudo yum -y install libxml2 libxml2-devel bison bison-devel boost-devel unixODBC-devel bzr perl perl\(DBI\)" \
  "sudo yum -y remove systemtap-sdt-dev" \
  "bzr co --lightweight lp:mariadb-native-client" \
  "sudo mkdir -vp /usr/src/redhat/SOURCES /usr/src/redhat/SPECS /usr/src/redhat/RPMS /usr/src/redhat/SRPMS" \
  "cd /usr/local/src;sudo tar zxf /dev/shm/thrift-0.9.0.tar.gz;pwd;ls" \
  "cd /usr/local/src/thrift-0.9.0;echo;pwd;sudo ./configure --prefix=/usr --enable-shared=no --enable-static=yes CXXFLAGS=-fPIC CFLAGS=-fPIC && echo && echo 'now making' && echo && sleep 3 && sudo make && echo && echo 'now installing' && echo && sleep 3 && sudo make install" ; \
done

VMs for install testing.

for i in '/kvm/vms/vm-fedora20-i386-serial.qcow2 2291 qemu64' '/kvm/vms/vm-fedora20-amd64-serial.qcow2 2292 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_install_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
    "sudo yum -y update" \
    "sudo yum -y install patch tar libaio perl perl-Time-HiRes perl-DBI unixODBC Judy" ; \
done

VMs for MariaDB upgrade testing (Fedora repo)

for i in '/kvm/vms/vm-fedora20-i386-serial.qcow2 2291 qemu64' '/kvm/vms/vm-fedora20-amd64-serial.qcow2 2292 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_upgrade_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
    "sudo yum -y update" \
    "sudo yum -y install patch tar mariadb-server libtool-ltdl unixODBC Judy" \
    "sudo systemctl enable mariadb.service" \
    "sudo systemctl start mariadb.service" \
    'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ; \
done

VMs for MariaDB upgrade testing (MariaDB repo)

for i in '/kvm/vms/vm-fedora20-amd64-serial.qcow2 2292 qemu64' '/kvm/vms/vm-fedora20-i386-serial.qcow2 2291 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_upgrade2_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
    'sudo yum -y update' \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB-${2}.repo buildbot@localhost:/tmp/MariaDB.repo" \
    'sudo rpm --verbose --import https://yum.mariadb.org/RPM-GPG-KEY-MariaDB' \
    'sudo mv -vi /tmp/MariaDB.repo /etc/yum.repos.d/' \
    'sudo yum -y remove mysql-libs' \
    'sudo yum -y install MariaDB-server MariaDB-client MariaDB-test libtool-ltdl unixODBC Judy' \
    'sudo yum -y install cronie cronie-anacron crontabs.noarch patch tar' \
    'sudo /etc/init.d/mysql start' \
    'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
    'sudo rm -v /etc/yum.repos.d/MariaDB.repo' \
    'sudo yum -y update' \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
    'sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/'; \
done

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Red Hat 6 x86

The following steps were used to create a Red Hat 6 x86 buildslave.

Initial Setup

cd vms
qemu-img create -f qcow2 vm-rhel6-x86-base.qcow2 8G
kvm -m 1024 -hda vm-rhel6-x86-base.qcow2 -cdrom ../iso/red-hat/rhel-server-6.0-i386-dvd.iso -redir 'tcp:22275::22' -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user

When the VM boots. Go through the prompts.

Re-initialize the drive, when prompted.

Set the Hostname to "rhel6-x86".

Configure Network, set eth0 to "Connect Automatically"

Set the root password.

Set partitioning type to: Use All Space

Don't encrypt the partitions (just adds overhead).

Select "Write Changes to Disk" on the popup that appears.

Set the software set to "Virtual Host".

After clicking next on the install-type page, the installation will finally start.

After installation completes, click reboot. Then shutdown the VM.

Serial Console Setup

cd vms
qemu-img create -b vm-rhel6-x86-base.qcow2 -f qcow2 vm-rhel6-x86-serial.qcow2
kvm -m 1024 -hda vm-rhel6-x86-serial.qcow2 -redir 'tcp:22275::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user

Add to /boot/grub/menu.lst:

serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial console

also add in menu.lst to kernel line (after removing 'quiet'):

console=tty0 console=ttyS0,115200n8

Add login prompt on serial console:

cat >>/etc/inittab <<END

# Serial console.
S0:2345:respawn:/sbin/agetty -h -L ttyS0 19200 vt100
END

Create buildbot account

With the network up and running, it's time to add a user so that we don't have to login as root all the time.

useradd buildbot
# Password is disabled by default in Centos5.
usermod -a -G wheel buildbot
visudo
# Uncomment the line "%wheel        ALL=(ALL)       NOPASSWD: ALL"
# Comment out this line:
# Defaults    requiretty

# Put in public ssh key for own account and host buildbot account.
# Note that Centos5 seems to require .ssh/authorized_keys chmod go-rwx.
su - buildbot
mkdir .ssh
chmod go-rwx .ssh

# scp the key from the vm host over to .ssh/authorized_keys

chmod go-rwx .ssh/authorized_keys

Now logout and then ssh to the VM as the buildbot user. On my local box I added the following to my /.ssh/config file to make logging in easier:

Host rhel6-x86
HostName localhost
  User buildbot
  IdentityFile ~/.ssh/id_rsa
  Port 2225

With the above in place I can simply type:

ssh rhel6-x86

...to connect to the vm.

RHN and Updates

sudo rhn_register

Choose defaults when registering. After the process is complete:

sudo yum update

The first time you update you'll be prompted to import some GPG keys from Red Hat. The updating process may take a while, depending on the number of updates and the speed of your Internet connection.

After updating shutdown so we can make more copies.

sudo shutdown -h now

Image for RPM Build

qemu-img create -b vm-rhel6-x86-serial.qcow2 -f qcow2 vm-rhel6-x86-build.qcow2
kvm -m 1024 -hda vm-rhel6-x86-build.qcow2 -redir 'tcp:22275::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

Wait for the system to boot.

Install compilers etc:

sudo yum groupinstall "Development Tools"
sudo yum install libaio-devel openssl-devel

If the "Development Tools" group is not available, the following lines will install the packages from it:

# Mandatory Packages:
sudo yum install autoconf automake binutils bison flex gcc gcc-c++ gettext
sudo yum install libtool make patch pkgconfig redhat-rpm-config rpm-build

# Default Packages:
sudo yum install byacc cscope ctags cvs diffstat doxygen elfutils gcc-gfortran
sudo yum install git indent intltool patchutils rcs subversion swig systemtap

# Optional Packages:
sudo yum install ElectricFence ant babel bzr chrpath cmake compat-gcc-34
sudo yum install compat-gcc-34-c++ compat-gcc-34-g77 dejagnu expect gcc-gnat
sudo yum install gcc-java gcc-objc gcc-objc++ imake jpackage-utils libstdc++-docs
sudo yum install mercurial mod_dav_svn nasm perltidy python-docs rpmdevtools
sudo yum install rpmlint systemtap-sdt-devel systemtap-server

Other packages to install:

sudo yum install gperf readline-devel ncurses-devel zlib-devel perl perl\(DBI\)

Create rpm directories and download 5.0 rpm for shared-compat:

sudo mkdir -vp /usr/src/redhat/SOURCES /usr/src/redhat/SPECS /usr/src/redhat/RPMS /usr/src/redhat/SRPMS
sudo mkdir -vp /srv/shared/yum/CentOS/5/i386/RPMS/
cd /srv/shared/yum/CentOS/5/i386/RPMS/
sudo wget http://mirror.ourdelta.org/yum/CentOS/5/i386/RPMS/MySQL-OurDelta-shared-5.0.87.d10-65.el5.i386.rpm

Image for install/test

qemu-img create -b vm-rhel6-x86-serial.qcow2 -f qcow2 vm-rhel6-x86-install.qcow2
kvm -m 1024 -hda vm-rhel6-x86-install.qcow2 -redir 'tcp:22275::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

Install extra dependencies:

sudo yum install perl perl\(DBI\)

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Ubuntu

Buildbot Setup for Virtual Machines - Ubuntu 10.04 (alpha), i386 and amd64

Base install

qemu-img create -f qcow2 vm-lucid-amd64-serial.qcow2 8G
kvm -m 1024 -hda vm-lucid-amd64-serial.qcow2 -cdrom /kvm/lucid-server-amd64.iso -redir tcp:2238::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-lucid-amd64-serial.qcow2 -cdrom /kvm/lucid-server-amd64.iso -redir tcp:2238::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2238 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2238 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2238 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2238 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2238 ttyS0.conf buildbot@localhost:
ssh -p 2238 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

qemu-img create -f qcow2 vm-lucid-i386-serial.qcow2 8G
kvm -m 1024 -hda vm-lucid-i386-serial.qcow2 -cdrom /kvm/lucid-server-i386.iso -redir tcp:2239::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-lucid-i386-serial.qcow2 -cdrom /kvm/lucid-server-i386.iso -redir tcp:2239::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
ssh -p 2239 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2239 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2239 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2239 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2239 ttyS0.conf buildbot@localhost:
ssh -p 2239 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

Enabling passwordless sudo:

sudo VISUAL=vi visudo
# uncomment `%sudo ALL=NOPASSWD: ALL' line in `visudo`, and move to end.

Editing /boot/grub/menu.lst:

sudo vi /etc/default/grub

# Add/edit these entries:
    GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
    GRUB_TERMINAL="serial"
    GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

sudo update-grub

VMs for building .debs

for i in 'vm-lucid-amd64-serial.qcow2 2238 qemu64' 'vm-lucid-i386-serial.qcow2 2239 qemu32,-nx' ; do \
  set $i; \
  runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.1" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline5-dev" ; \
done

VMs for install testing.

See above for how to obtain my.seed and sources.append.

cat >sources.append <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END

for i in 'vm-lucid-amd64-serial.qcow2 2238 qemu64' 'vm-lucid-i386-serial.qcow2 2239 qemu32,-nx' ; do \
  set $i; \
  runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y debconf-utils" \
    "= scp -P $2 my.seed sources.append buildbot@localhost:/tmp/" \
    "sudo debconf-set-selections /tmp/my.seed" \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
done

VMs for upgrade testing

for i in 'vm-lucid-amd64-install.qcow2 2238 qemu64' 'vm-lucid-i386-install.qcow2 2239 qemu32,-nx' ; do \
  set $i; \
  runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.1' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
done

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Ubuntu 10.10 "maverick"

Base install

qemu-img create -f qcow2 vm-maverick-amd64-serial.qcow2 8G
kvm -m 1024 -hda vm-maverick-amd64-serial.qcow2 -cdrom /kvm/ubuntu-10.10-server-amd64.iso -redir tcp:2246::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-maverick-amd64-serial.qcow2 -cdrom /kvm/ubuntu-10.10-server-amd64.iso -redir tcp:2246::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2246 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2246 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2246 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2246 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2246 ttyS0.conf buildbot@localhost:
ssh -p 2246 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

qemu-img create -f qcow2 vm-maverick-i386-serial.qcow2 8G
kvm -m 1024 -hda vm-maverick-i386-serial.qcow2 -cdrom /kvm/ubuntu-10.10-server-i386.iso -redir tcp:2247::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-maverick-i386-serial.qcow2 -cdrom /kvm/ubuntu-10.10-server-i386.iso -redir tcp:2247::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
ssh -p 2247 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2247 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2247 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2247 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2247 ttyS0.conf buildbot@localhost:
ssh -p 2247 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

Enabling passwordless sudo:

sudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'

Editing /boot/grub/menu.lst:

sudo vi /etc/default/grub

# Add/edit these entries:
    GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
    GRUB_TERMINAL="serial"
    GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

sudo update-grub

VMs for building .debs

for i in 'vm-maverick-amd64-serial.qcow2 2246 qemu64' 'vm-maverick-i386-serial.qcow2 2247 qemu32,-nx' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.1" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline6-dev" ; \
done

VMs for install testing.

See the General Principles article for how to make the 'my.seed' and 'sources.append' files.

cat >sources.append <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END

for i in 'vm-maverick-amd64-serial.qcow2 2246 qemu64' 'vm-maverick-i386-serial.qcow2 2247 qemu32,-nx' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y debconf-utils" \
    "= scp -P $2 my.seed sources.append buildbot@localhost:/tmp/" \
    "sudo debconf-set-selections /tmp/my.seed" \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
done

VMs for upgrade testing

for i in 'vm-maverick-amd64-install.qcow2 2246 qemu64' 'vm-maverick-i386-install.qcow2 2247 qemu32,-nx' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.1' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
done

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Ubuntu 11.04 "natty"

Base install

qemu-img create -f qcow2 vm-natty-amd64-serial.qcow2 8G
kvm -m 1024 -hda vm-natty-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.04-server-amd64.iso -redir tcp:2255::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

# Install, picking default options mostly, only adding openssh server.

kvm -m 1024 -hda vm-natty-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.04-server-amd64.iso -redir tcp:2255::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2255 localhost

# edit /boot/grub/menu.lst and visudo, see below

ssh -t -p 2255 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2255 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2255 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2255 ttyS0.conf buildbot@localhost:
ssh -p 2255 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

qemu-img create -f qcow2 vm-natty-i386-serial.qcow2 8G
kvm -m 1024 -hda vm-natty-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.04-server-i386.iso -redir tcp:2256::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user

# Install, picking default options mostly, only adding openssh server.

kvm -m 1024 -hda vm-natty-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.04-server-i386.iso -redir tcp:2256::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2256 localhost

# edit /boot/grub/menu.lst and visudo, see below

ssh -t -p 2256 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2256 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2256 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2256 ttyS0.conf buildbot@localhost:
ssh -p 2256 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

Enabling passwordless sudo:

sudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'

Editing /boot/grub/menu.lst:

sudo vi /etc/default/grub

# Add/edit these entries:
    GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
    GRUB_TERMINAL="serial"
    GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

sudo update-grub

VMs for building .debs

for i in 'vm-natty-amd64-serial.qcow2 2255 qemu64' 'vm-natty-i386-serial.qcow2 2256 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.1" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline6-dev" ; \
done

VMs for install testing.

See above for how to obtain my.seed and sources.append.

for i in 'vm-natty-amd64-serial.qcow2 2255 qemu64' 'vm-natty-i386-serial.qcow2 2256 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y debconf-utils" \
    "= scp -P $2 my.seed sources.append buildbot@localhost:/tmp/" \
    "sudo debconf-set-selections /tmp/my.seed" \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
done

VMs for upgrade testing

for i in 'vm-natty-amd64-install.qcow2 2255 qemu64' 'vm-natty-i386-install.qcow2 2256 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.1' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
done

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Ubuntu 11.10 "oneiric"

Base install

qemu-img create -f qcow2 /kvm/vms/vm-oneiric-amd64-serial.qcow2 8G
qemu-img create -f qcow2 /kvm/vms/vm-oneiric-i386-serial.qcow2 8G

# Start each VM booting from the server install iso one at a time and perform
# the following install steps:
kvm -m 1024 -hda /kvm/vms/vm-oneiric-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.10-server-amd64.iso -redir tcp:2255::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 1024 -hda /kvm/vms/vm-oneiric-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.10-server-i386.iso -redir tcp:2256::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user

# Once running you can connect to the VNC server from your local host with:
#
#    vncviewer -via ${remote-host} localhost
#
# (replace ${remote-host} with the host the vm is running on)

# Install, picking default options mostly, with the following notes:
# * Set the hostname to ubuntu-oneiric
# * When partitioning disks, choose "Guided - use entire disk" (we do not want LVM)
# * No automatic updates
# * Choose software to install: OpenSSH server


# Now that the VM is installed, it's time to configure it.
# If you have the memory you can do the following simultaneously:
kvm -m 1024 -hda /kvm/vms/vm-oneiric-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.10-server-amd64.iso -redir tcp:2255::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
kvm -m 1024 -hda /kvm/vms/vm-oneiric-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.10-server-i386.iso -redir tcp:2256::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

ssh -p 2255 localhost
# edit /boot/grub/menu.lst and visudo, see below

ssh -p 2256 localhost
# edit /boot/grub/menu.lst and visudo, see below

ssh -t -p 2255 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
ssh -t -p 2256 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"

scp -P 2255 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2256 /kvm/vms/authorized_keys localhost:.ssh/

echo $'Buildbot\n\n\n\n\ny' | ssh -p 2255 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2256 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'

scp -P 2255 /kvm/vms/ttyS0.conf buildbot@localhost:
scp -P 2256 /kvm/vms/ttyS0.conf buildbot@localhost:

ssh -p 2255 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'
ssh -p 2256 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'

Enabling passwordless sudo:

sudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'

Editing /boot/grub/menu.lst:

sudo vi /etc/default/grub

# Add/edit these entries:
    GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
    GRUB_TERMINAL="serial"
    GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

sudo update-grub

VMs for building .debs

for i in '/kvm/vms/vm-oneiric-amd64-serial.qcow2 2255 qemu64' '/kvm/vms/vm-oneiric-i386-serial.qcow2 2256 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.1" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline6-dev" ; \
done

Install cmake (required for MariaDB 5.5) with:

kvm -m 1024 -hda ${vm} -redir tcp:22565::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

ssh -o IdentityFile=.ssh/buildbot.id_dsa -p 22565 buildbot@localhost

wget http://www.cmake.org/files/v2.8/cmake-2.8.6.tar.gz
tar -zxvf cmake-2.8.6.tar.gz
cd cmake-2.8.6;./configure
make
sudo make install

sudo /sbin/shutdown -h now

Also:

Installing the Boost library needed for the OQGraph storage engine

VMs for install testing.

See above for how to obtain my.seed and sources.append.

for i in '/kvm/vms/vm-oneiric-amd64-serial.qcow2 2255 qemu64' '/kvm/vms/vm-oneiric-i386-serial.qcow2 2256 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y debconf-utils" \
    "= scp -P $2 /kvm/vms/my.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
    "sudo debconf-set-selections /tmp/my.seed" \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
done

VMs for MySQL upgrade testing

for i in '/kvm/vms/vm-oneiric-amd64-install.qcow2 2255 qemu64' '/kvm/vms/vm-oneiric-i386-install.qcow2 2256 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.1' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
done

VMs for MariaDB upgrade testing

The steps below are based on the Natty steps on Installing VM images for testing .deb upgrade between versions.

64-bit Ubuntu oneiric:

qemu-img create -b vm-oneiric-amd64-install.qcow2 -f qcow2 vm-oneiric-amd64-upgrade2.qcow2
kvm -m 512 -hda vm-oneiric-amd64-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

sudo sh -c 'cat > /etc/apt/sources.list.d/tmp.list'
deb http://ftp.osuosl.org/pub/mariadb/repo/5.2/ubuntu natty main
deb-src http://ftp.osuosl.org/pub/mariadb/repo/5.2/ubuntu natty main

sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 1BB943DB
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
sudo rm /etc/apparmor.d/usr.sbin.mysqld
sudo /etc/init.d/apparmor restart
sudo apt-get -f install
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
sudo rm /etc/apt/sources.list.d/tmp.list
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y
sudo shutdown -h now

32-bit Ubuntu oneiric:

qemu-img create -b vm-oneiric-i386-install.qcow2 -f qcow2 vm-oneiric-i386-upgrade2.qcow2
kvm -m 512 -hda vm-oneiric-i386-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

sudo sh -c 'cat > /etc/apt/sources.list.d/tmp.list'
deb http://ftp.osuosl.org/pub/mariadb/repo/5.2/ubuntu natty main
deb-src http://ftp.osuosl.org/pub/mariadb/repo/5.2/ubuntu natty main

sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 1BB943DB
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
sudo rm /etc/apparmor.d/usr.sbin.mysqld
sudo /etc/init.d/apparmor restart
sudo apt-get -f install
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
sudo rm /etc/apt/sources.list.d/tmp.list
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y
sudo shutdown -h now

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Ubuntu 12.04 "precise"

Base install

qemu-img create -f qcow2 /kvm/vms/vm-precise-amd64-serial.qcow2 8G
qemu-img create -f qcow2 /kvm/vms/vm-precise-i386-serial.qcow2 8G

Start each VM booting from the server install iso one at a time and perform the following install steps:

kvm -m 1024 -hda /kvm/vms/vm-precise-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.04-server-amd64.iso -redir tcp:22255::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 1024 -hda /kvm/vms/vm-precise-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.04-server-i386.iso -redir tcp:22256::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user

Once running you can connect to the VNC server from your local host with:

vncviewer -via ${remote-host} localhost

Replace ${remote-host} with the host the vm is running on.

Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. Ubuntu has just resized the vnc screen. Simply reconnect.

Install, picking default options mostly, with the following notes:

Set the hostname to ubuntu-precise
When partitioning disks, choose "Guided - use entire disk" (we do not want LVM)
No automatic updates
Choose software to install: OpenSSH server

Now that the VM is installed, it's time to configure it. If you have the memory you can do the following simultaneously:

kvm -m 1024 -hda /kvm/vms/vm-precise-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.04-server-amd64.iso -redir tcp:22255::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
kvm -m 1024 -hda /kvm/vms/vm-precise-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.04-server-i386.iso -redir tcp:22256::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

ssh -p 22255 localhost
# edit /boot/grub/menu.lst and visudo, see below

ssh -p 22256 localhost
# edit /boot/grub/menu.lst and visudo, see below

ssh -t -p 22255 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
ssh -t -p 22256 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"

scp -P 22255 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 22256 /kvm/vms/authorized_keys localhost:.ssh/

echo $'Buildbot\n\n\n\n\ny' | ssh -p 22255 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
echo $'Buildbot\n\n\n\n\ny' | ssh -p 22256 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'

scp -P 22255 /kvm/vms/ttyS0.conf buildbot@localhost:
scp -P 22256 /kvm/vms/ttyS0.conf buildbot@localhost:

ssh -p 22255 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'
ssh -p 22256 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'

Enabling passwordless sudo:

sudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'

Editing /boot/grub/menu.lst:

sudo vi /etc/default/grub

# Add/edit these entries:
    GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
    GRUB_TERMINAL="serial"
    GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

sudo update-grub

VMs for building .debs

for i in '/kvm/vms/vm-precise-amd64-serial.qcow2 22255 qemu64' '/kvm/vms/vm-precise-i386-serial.qcow2 22256 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.5" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libpam0g-dev libreadline-gplv2-dev autoconf automake automake1.9 defoma dpatch ghostscript-x libfontenc1 libjpeg62 libltdl-dev libltdl7 libmail-sendmail-perl libxfont1 lmodern psfontmgr texlive-latex-base-doc ttf-dejavu ttf-dejavu-extra libaio-dev xfonts-encodings xfonts-utils" ; \
done

Also:

Installing the Boost library needed for the OQGraph storage engine

VMs for install testing.

See Buildbot Setup for Virtual Machines - General Principles for how to obtain my.seed and sources.append.

for i in '/kvm/vms/vm-precise-amd64-serial.qcow2 22255 qemu64' '/kvm/vms/vm-precise-i386-serial.qcow2 22256 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils" \
    "= scp -P $2 /kvm/vms/my.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
    "sudo debconf-set-selections /tmp/my.seed" \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
done

VMs for MySQL upgrade testing

for i in '/kvm/vms/vm-precise-amd64-install.qcow2 22255 qemu64' '/kvm/vms/vm-precise-i386-install.qcow2 22256 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y libaio1 mysql-server-5.5' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
done

VMs for MariaDB upgrade testing

The steps below are based on the Natty steps on Installing VM images for testing .deb upgrade between versions.

64-bit Ubuntu precise:

qemu-img create -b vm-precise-amd64-install.qcow2 -f qcow2 vm-precise-amd64-upgrade2.qcow2
kvm -m 512 -hda vm-precise-amd64-upgrade2.qcow2 -redir 'tcp:22200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 22200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

sudo sh -c 'cat > /etc/apt/sources.list.d/tmp.list'
deb http://ftp.osuosl.org/pub/mariadb/repo/5.5/ubuntu oneiric main
deb-src http://ftp.osuosl.org/pub/mariadb/repo/5.5/ubuntu oneiric main

sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 1BB943DB
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
sudo apt-get -f install
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
sudo rm /etc/apt/sources.list.d/tmp.list
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y libaio1
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y
sudo shutdown -h now

32-bit Ubuntu precise:

qemu-img create -b vm-precise-i386-install.qcow2 -f qcow2 vm-precise-i386-upgrade2.qcow2
kvm -m 512 -hda vm-precise-i386-upgrade2.qcow2 -redir 'tcp:22200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 22200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

sudo sh -c 'cat > /etc/apt/sources.list.d/tmp.list'
deb http://ftp.osuosl.org/pub/mariadb/repo/5.5/ubuntu oneiric main
deb-src http://ftp.osuosl.org/pub/mariadb/repo/5.5/ubuntu oneiric main

sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 1BB943DB
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
sudo apt-get -f install
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
sudo rm /etc/apt/sources.list.d/tmp.list
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y libaio1
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y
sudo shutdown -h now

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Ubuntu 12.10 "quantal"

Base install

qemu-img create -f qcow2 /kvm/vms/vm-quantal-amd64-serial.qcow2 10G
qemu-img create -f qcow2 /kvm/vms/vm-quantal-i386-serial.qcow2 10G

Start each VM booting from the server install iso one at a time and perform the following install steps:

kvm -m 2048 -hda /kvm/vms/vm-quantal-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.10-server-amd64.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2275-:22
kvm -m 2048 -hda /kvm/vms/vm-quantal-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.10-server-i386.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2276-:22

Once running you can connect to the VNC server from your local host with:

vncviewer -via ${remote-host} localhost

Replace ${remote-host} with the host the vm is running on.

Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. Ubuntu has just resized the vnc screen. Simply reconnect.

Install, picking default options mostly, with the following notes:

Set the hostname to ubuntu-quantal-amd64 or ubuntu-quantal-i386
When partitioning disks, choose "Guided - use entire disk" (we do not want LVM)
No automatic updates
Choose software to install: OpenSSH server

Now that the VM is installed, it's time to configure it. If you have the memory you can do the following simultaneously:

kvm -m 2048 -hda /kvm/vms/vm-quantal-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.10-server-amd64.iso -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2275-:22 -nographic
kvm -m 2048 -hda /kvm/vms/vm-quantal-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.10-server-i386.iso -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2276-:22 -nographic

ssh -p 2275 localhost
# edit /boot/grub/menu.lst and visudo, see below

ssh -p 2276 localhost
# edit /boot/grub/menu.lst and visudo, see below

ssh -t -p 2275 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
ssh -t -p 2276 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"

scp -P 2275 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2276 /kvm/vms/authorized_keys localhost:.ssh/

echo $'Buildbot\n\n\n\n\ny' | ssh -p 2275 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2276 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'

scp -P 2275 /kvm/vms/ttyS0.conf buildbot@localhost:
scp -P 2276 /kvm/vms/ttyS0.conf buildbot@localhost:

ssh -p 2275 buildbot@localhost 'sudo apt-get update && sudo apt-get -y dist-upgrade;'
ssh -p 2276 buildbot@localhost 'sudo apt-get update && sudo apt-get -y dist-upgrade;'

ssh -p 2275 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'
ssh -p 2276 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'

Enabling passwordless sudo:

sudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'

Editing /boot/grub/menu.lst:

sudo vi /etc/default/grub

# Add/edit these entries:
    GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
    GRUB_TERMINAL="serial"
    GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

sudo update-grub
# exit back to the host server

VMs for building .debs

for i in '/kvm/vms/vm-quantal-amd64-serial.qcow2 2275 qemu64' '/kvm/vms/vm-quantal-i386-serial.qcow2 2276 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/boost_1_49_0.tar.gz buildbot@localhost:/dev/shm/" \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/thrift-0.9.0.tar.gz buildbot@localhost:/dev/shm/" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.5" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libpam0g-dev libreadline-gplv2-dev autoconf automake automake1.9 defoma dpatch ghostscript-x libfontenc1 libjpeg62 libltdl-dev libltdl7 libmail-sendmail-perl libxfont1 lmodern psfontmgr texlive-latex-base-doc ttf-dejavu ttf-dejavu-extra libaio-dev xfonts-encodings xfonts-utils libxml2-dev unixodbc-dev" \
    "cd /usr/local/src;sudo tar zxf /dev/shm/thrift-0.9.0.tar.gz;pwd;ls" \
    "cd /usr/local/src/thrift-0.9.0;echo;pwd;sudo ./configure --prefix=/usr --enable-shared=no --enable-static=yes CXXFLAGS=-fPIC CFLAGS=-fPIC && echo && echo 'now making' && echo && sleep 3 && sudo make && echo && echo 'now installing' && echo && sleep 3 && sudo make install" \
    "cd /usr/local/src;sudo tar zxf /dev/shm/boost_1_49_0.tar.gz;cd /usr/local/include/;sudo ln -vs ../src/boost_1_49_0/boost ." ; \
done

VMs for install testing.

See Buildbot Setup for Virtual Machines - General Principles for how to obtain my.seed and sources.append.

for i in '/kvm/vms/vm-quantal-amd64-serial.qcow2 2275 qemu64' '/kvm/vms/vm-quantal-i386-serial.qcow2 2276 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils" \
    "= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
    "sudo debconf-set-selections /tmp/my55.seed" \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
done

VMs for MySQL upgrade testing

for i in '/kvm/vms/vm-quantal-amd64-serial.qcow2 2275 qemu64' '/kvm/vms/vm-quantal-i386-serial.qcow2 2276 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils" \
    "= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
    "sudo debconf-set-selections /tmp/my55.seed" \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.5' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
done

VMs for MariaDB upgrade testing

The steps below are based on the Natty steps on Installing VM images for testing .deb upgrade between versions.

for i in '/kvm/vms/vm-quantal-amd64-serial.qcow2 2275 qemu64' '/kvm/vms/vm-quantal-i386-serial.qcow2 2276 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
    "= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
    "= scp -P $2 /kvm/vms/mariadb-quantal.list buildbot@localhost:/tmp/tmp.list" \
    "sudo debconf-set-selections /tmp/my55.seed" \
    'sudo mv -vi /tmp/tmp.list /etc/apt/sources.list.d/' \
    'sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 0xcbcb082a1bb943db' \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mariadb-server mariadb-server-5.5 mariadb-client mariadb-client-5.5 mariadb-test libmariadbclient-dev' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
    'sudo rm -v /etc/apt/sources.list.d/tmp.list' \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get update' \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils' \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y'; \
done

Add Key to known_hosts

Do the following on each kvm host server (terrier, terrier2, i7, etc...) to add the VMs to known_hosts.

# quantal-amd64
cp -avi /kvm/vms/vm-quantal-amd64-install.qcow2 /kvm/vms/vm-quantal-amd64-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-quantal-amd64-test.qcow2 -redir tcp:2275::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2275 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-quantal-amd64-test.qcow2

# quantal-i386
cp -avi /kvm/vms/vm-quantal-i386-install.qcow2 /kvm/vms/vm-quantal-i386-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-quantal-i386-test.qcow2 -redir tcp:2276::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2276 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-quantal-i386-test.qcow2

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Ubuntu 13.04 "raring"

Base install

qemu-img create -f qcow2 /kvm/vms/vm-raring-amd64-serial.qcow2 15G
qemu-img create -f qcow2 /kvm/vms/vm-raring-i386-serial.qcow2 15G

Start each VM booting from the server install iso one at a time and perform the following install steps:

kvm -m 2048 -hda /kvm/vms/vm-raring-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.04-server-amd64.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2279-:22
kvm -m 2048 -hda /kvm/vms/vm-raring-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.04-server-i386.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2280-:22

Once running you can connect to the VNC server from your local host with:

vncviewer -via ${remote-host} localhost

Replace ${remote-host} with the host the vm is running on.

Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. Ubuntu has just resized the vnc screen. Simply reconnect.

Install, picking default options mostly, with the following notes:

Set the hostname to ubuntu-raring-amd64 or ubuntu-raring-i386
When partitioning disks, choose "Guided - use entire disk" (we do not want LVM)
No automatic updates
Choose software to install: OpenSSH server

Now that the VM is installed, it's time to configure it. If you have the memory you can do the following simultaneously:

kvm -m 2048 -hda /kvm/vms/vm-raring-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.04-server-amd64.iso -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2279-:22 -nographic
kvm -m 2048 -hda /kvm/vms/vm-raring-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.04-server-i386.iso -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2280-:22 -nographic

ssh -p 2279 localhost
# edit /boot/grub/menu.lst and visudo, see below

ssh -p 2280 localhost
# edit /boot/grub/menu.lst and visudo, see below

ssh -t -p 2279 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
ssh -t -p 2280 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"

scp -P 2279 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2280 /kvm/vms/authorized_keys localhost:.ssh/

echo $'Buildbot\n\n\n\n\ny' | ssh -p 2279 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2280 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'

scp -P 2279 /kvm/vms/ttyS0.conf buildbot@localhost:
scp -P 2280 /kvm/vms/ttyS0.conf buildbot@localhost:

ssh -p 2279 buildbot@localhost 'sudo apt-get update && sudo apt-get -y dist-upgrade;'
ssh -p 2280 buildbot@localhost 'sudo apt-get update && sudo apt-get -y dist-upgrade;'

ssh -p 2279 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'
ssh -p 2280 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'

Enabling passwordless sudo:

sudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'

Editing /boot/grub/menu.lst:

sudo vi /etc/default/grub

# Add/edit these entries:
GRUB_HIDDEN_TIMEOUT_QUIET=false
GRUB_TIMEOUT=0
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

sudo update-grub
# exit back to the host server

VMs for building .debs

for i in '/kvm/vms/vm-raring-amd64-serial.qcow2 2279 qemu64' '/kvm/vms/vm-raring-i386-serial.qcow2 2280 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/boost_1_49_0.tar.gz buildbot@localhost:/dev/shm/" \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/thrift-0.9.0.tar.gz buildbot@localhost:/dev/shm/" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.5" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libpam0g-dev libreadline-gplv2-dev autoconf automake automake1.9 dpatch ghostscript-x libfontenc1 libjpeg62 libltdl-dev libltdl7 libmail-sendmail-perl libxfont1 lmodern texlive-latex-base-doc ttf-dejavu ttf-dejavu-extra libaio-dev xfonts-encodings xfonts-utils libxml2-dev unixodbc-dev bzr" \
    "bzr co --lightweight lp:mariadb-native-client" \
    "cd /usr/local/src;sudo tar zxf /dev/shm/thrift-0.9.0.tar.gz;pwd;ls" \
    "cd /usr/local/src/thrift-0.9.0;echo;pwd;sudo ./configure --prefix=/usr --enable-shared=no --enable-static=yes CXXFLAGS=-fPIC CFLAGS=-fPIC && echo && echo 'now making' && echo && sleep 3 && sudo make && echo && echo 'now installing' && echo && sleep 3 && sudo make install" \
    "cd /usr/local/src;sudo tar zxf /dev/shm/boost_1_49_0.tar.gz;cd /usr/local/include/;sudo ln -vs ../src/boost_1_49_0/boost ." ; \
done

VMs for install testing.

See Buildbot Setup for Virtual Machines - General Principles for how to obtain my.seed and sources.append.

for i in '/kvm/vms/vm-raring-amd64-serial.qcow2 2279 qemu64' '/kvm/vms/vm-raring-i386-serial.qcow2 2280 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils" \
    "= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
    "sudo debconf-set-selections /tmp/my55.seed" \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
done

VMs for MySQL upgrade testing

for i in '/kvm/vms/vm-raring-amd64-serial.qcow2 2279 qemu64' '/kvm/vms/vm-raring-i386-serial.qcow2 2280 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils" \
    "= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
    "sudo debconf-set-selections /tmp/my55.seed" \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.5' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
done

VMs for MariaDB upgrade testing

The steps below are based on the Natty steps on Installing VM images for testing .deb upgrade between versions.

for i in '/kvm/vms/vm-raring-amd64-serial.qcow2 2279 qemu64' '/kvm/vms/vm-raring-i386-serial.qcow2 2280 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
    "= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
    "= scp -P $2 /kvm/vms/mariadb-raring.list buildbot@localhost:/tmp/tmp.list" \
    "sudo debconf-set-selections /tmp/my55.seed" \
    'sudo mv -vi /tmp/tmp.list /etc/apt/sources.list.d/' \
    'sudo apt-key adv --recv-keys --keyserver pgp.mit.edu 0xcbcb082a1bb943db' \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mariadb-server mariadb-server-5.5 mariadb-client mariadb-client-5.5 mariadb-test libmariadbclient-dev mysql-common=5.5.30-mariadb1~quantal libmysqlclient18=5.5.30-mariadb1~quantal' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
    'sudo rm -v /etc/apt/sources.list.d/tmp.list' \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get update' \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils' \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y'; \
done

Add Key to known_hosts

Do the following on each kvm host server (terrier, terrier2, i7, etc...) to add the VMs to known_hosts.

# raring-amd64
cp -avi /kvm/vms/vm-raring-amd64-install.qcow2 /kvm/vms/vm-raring-amd64-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-raring-amd64-test.qcow2 -redir tcp:2279::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2279 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-raring-amd64-test.qcow2

# raring-i386
cp -avi /kvm/vms/vm-raring-i386-install.qcow2 /kvm/vms/vm-raring-i386-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-raring-i386-test.qcow2 -redir tcp:2280::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2280 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-raring-i386-test.qcow2

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Ubuntu 13.10 "saucy"

Base install

qemu-img create -f qcow2 /kvm/vms/vm-saucy-amd64-serial.qcow2 20G
qemu-img create -f qcow2 /kvm/vms/vm-saucy-i386-serial.qcow2 20G

Start each VM booting from the server install iso one at a time and perform the following install steps:

kvm -m 2048 -hda /kvm/vms/vm-saucy-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.10-server-amd64.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2287-:22
kvm -m 2048 -hda /kvm/vms/vm-saucy-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.10-server-i386.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2288-:22

Once running you can connect to the VNC server from your local host with:

vncviewer -via ${remote_host} localhost

Replace ${remote-host} with the host the vm is running on.

Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. Ubuntu has just resized the vnc screen. Simply reconnect.

Install, picking default options mostly, with the following notes:

Set the hostname to ubuntu-saucy-amd64 or ubuntu-saucy-i386
do not encrypt the home directory
When partitioning disks, choose "Guided - use entire disk" (we do not want LVM)
No automatic updates
Choose software to install: OpenSSH server

Now that the VM is installed, it's time to configure it. If you have the memory you can do the following simultaneously:

kvm -m 2048 -hda /kvm/vms/vm-saucy-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.10-server-amd64.iso -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2287-:22 -nographic
kvm -m 2048 -hda /kvm/vms/vm-saucy-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.10-server-i386.iso -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2288-:22 -nographic

ssh -p 2287 localhost
sudo update-alternatives --config editor
# edit /boot/grub/menu.lst and visudo, see below

ssh -p 2288 localhost
sudo update-alternatives --config editor
# edit /boot/grub/menu.lst and visudo, see below


ssh -t -p 2287 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
ssh -t -p 2288 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"

scp -P 2287 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2288 /kvm/vms/authorized_keys localhost:.ssh/

echo $'Buildbot\n\n\n\n\ny' | ssh -p 2287 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2288 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'

scp -P 2287 /kvm/vms/ttyS0.conf buildbot@localhost:
scp -P 2288 /kvm/vms/ttyS0.conf buildbot@localhost:

ssh -p 2287 buildbot@localhost 'sudo apt-get update && sudo apt-get -y dist-upgrade;'
ssh -p 2288 buildbot@localhost 'sudo apt-get update && sudo apt-get -y dist-upgrade;'

ssh -p 2287 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'
ssh -p 2288 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'

Enabling passwordless sudo:

sudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'

Editing /boot/grub/menu.lst:

sudo vi /etc/default/grub

# Add/edit these entries:
GRUB_HIDDEN_TIMEOUT_QUIET=false
GRUB_TIMEOUT=0
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

sudo update-grub
# exit back to the host server

VMs for building .debs

for i in '/kvm/vms/vm-saucy-amd64-serial.qcow2 2287 qemu64' '/kvm/vms/vm-saucy-i386-serial.qcow2 2288 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/thrift-0.9.0.tar.gz buildbot@localhost:/dev/shm/" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.5" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libpam0g-dev libreadline-gplv2-dev autoconf automake automake1.9 dpatch ghostscript-x libfontenc1 libjpeg62 libltdl-dev libltdl7 libmail-sendmail-perl libxfont1 lmodern texlive-latex-base-doc ttf-dejavu ttf-dejavu-extra libaio-dev xfonts-encodings xfonts-utils libxml2-dev unixodbc-dev bzr scons check libboost-all-dev openssl epm" \
    "bzr co --lightweight lp:mariadb-native-client" \
    "cd /usr/local/src;sudo tar zxf /dev/shm/thrift-0.9.0.tar.gz;pwd;ls" \
    "cd /usr/local/src/thrift-0.9.0;echo;pwd;sudo ./configure --prefix=/usr --enable-shared=no --enable-static=yes CXXFLAGS=-fPIC CFLAGS=-fPIC && echo && echo 'now making' && echo && sleep 3 && sudo make && echo && echo 'now installing' && echo && sleep 3 && sudo make install" ; \
done

VMs for install testing.

See Buildbot Setup for Virtual Machines - General Principles for how to obtain my.seed and sources.append.

for i in '/kvm/vms/vm-saucy-amd64-serial.qcow2 2287 qemu64' '/kvm/vms/vm-saucy-i386-serial.qcow2 2288 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils unixodbc libxml2" \
    "= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
    "sudo debconf-set-selections /tmp/my55.seed" \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
done

VMs for MySQL upgrade testing

for i in '/kvm/vms/vm-saucy-amd64-serial.qcow2 2287 qemu64' '/kvm/vms/vm-saucy-i386-serial.qcow2 2288 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils" \
    "= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
    "sudo debconf-set-selections /tmp/my55.seed" \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.5' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
done

VMs for MariaDB upgrade testing

for i in '/kvm/vms/vm-saucy-amd64-serial.qcow2 2287 qemu64' '/kvm/vms/vm-saucy-i386-serial.qcow2 2288 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
    "= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
    "= scp -P $2 /kvm/vms/mariadb-saucy.list buildbot@localhost:/tmp/tmp.list" \
    "sudo debconf-set-selections /tmp/my55.seed" \
    'sudo mv -vi /tmp/tmp.list /etc/apt/sources.list.d/' \
    'sudo apt-key adv --recv-keys --keyserver pgp.mit.edu 0xcbcb082a1bb943db' \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mariadb-server' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
    'sudo rm -v /etc/apt/sources.list.d/tmp.list' \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get update' \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils' \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y'; \
done

Add Key to known_hosts

Do the following on each kvm host server (terrier, terrier2, i7, etc...) to add the VMs to known_hosts.

# saucy-amd64
cp -avi /kvm/vms/vm-saucy-amd64-install.qcow2 /kvm/vms/vm-saucy-amd64-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-saucy-amd64-test.qcow2 -redir tcp:2287::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2287 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-saucy-amd64-test.qcow2

# saucy-i386
cp -avi /kvm/vms/vm-saucy-i386-install.qcow2 /kvm/vms/vm-saucy-i386-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-saucy-i386-test.qcow2 -redir tcp:2288::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2288 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-saucy-i386-test.qcow2

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Ubuntu 14.04 "trusty"

Base install

qemu-img create -f qcow2 /kvm/vms/vm-trusty-amd64-serial.qcow2 20G
qemu-img create -f qcow2 /kvm/vms/vm-trusty-i386-serial.qcow2 20G

Start each VM booting from the server install iso one at a time and perform the following install steps:

kvm -m 2048 -hda /kvm/vms/vm-trusty-amd64-serial.qcow2 \
  -cdrom /kvm/iso/ubuntu/trusty-server-amd64.iso \
  -boot d -smp 2 -cpu qemu64 \
  -net nic,model=virtio \
  -net user,hostfwd=tcp:127.0.0.1:2293-:22

kvm -m 2048 -hda /kvm/vms/vm-trusty-i386-serial.qcow2 \
  -cdrom /kvm/iso/ubuntu/trusty-server-i386.iso \
  -boot d -smp 2 -cpu qemu64 \
  -net nic,model=virtio \
  -net user,hostfwd=tcp:127.0.0.1:2294-:22

Once running you can connect to the VNC server from your local host with:

vncviewer -via ${remote_host} localhost

Replace ${remote-host} with the host the vm is running on.

Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. Ubuntu has just resized the vnc screen. Simply reconnect.

During the install, pick default options for the most part, with the following notes:

Set the hostname to ubuntu-trusty-amd64 or ubuntu-trusty-i386 (depending on which architecture we're installing)
do not encrypt the home directory
When prompted if the timezone is correct, answer "No" and when the list comes up, scroll to the bottom and choose "UTC"
When partitioning disks, choose "Guided - use entire disk" (we do not want LVM)
No automatic updates
Choose software to install: OpenSSH server

Reboot when prompted. It will fail, so just kill the kvm process with^Ctrl+C.

Now that the VM is installed, it's time to configure it. If you have the memory you can do the following simultaneously:

qemu-system-x86_64 -m 2048 -hda /kvm/vms/vm-trusty-amd64-serial.qcow2 \
  -boot c -smp 2 -cpu qemu64 \
  -net nic,model=virtio \
  -net user,hostfwd=tcp:127.0.0.1:2293-:22 \
  -nographic

qemu-system-x86_64 -m 2048 -hda /kvm/vms/vm-trusty-i386-serial.qcow2 \
  -boot c -smp 2 -cpu qemu64 \
  -net nic,model=virtio \
  -net user,hostfwd=tcp:127.0.0.1:2294-:22 \
  -nographic

set vim as the preferred editor

ssh -t -p 2293 localhost sudo update-alternatives --config editor
ssh -t -p 2294 localhost sudo update-alternatives --config editor

Enabling passwordless sudo:

sudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'

edit /boot/grub/menu.lst:

sudo vi /etc/default/grub

# Add/edit these entries:
GRUB_HIDDEN_TIMEOUT_QUIET=false
GRUB_TIMEOUT=0
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

sudo update-grub
# exit back to the host server

create the .ssh directory and the sudo group

ssh -t -p 2293 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
ssh -t -p 2294 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"

copy the authorized_keys file up to the VMs

scp -P 2293 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2294 /kvm/vms/authorized_keys localhost:.ssh/

create the buildbot user on the amd64 VM

echo $'Buildbot\n\n\n\n\ny' | ssh -p 2293 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'

create the buildbot user on the x86 VM

echo $'Buildbot\n\n\n\n\ny' | ssh -p 2294 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'

copy the ttyS0 file to the VMs

scp -i -P 2293 /kvm/vms/ttyS0.conf buildbot@localhost:
scp -i -P 2294 /kvm/vms/ttyS0.conf buildbot@localhost:

Apply all OS updates

ssh -t -i -p 2293 buildbot@localhost \
  'sudo apt-get update && sudo apt-get -y dist-upgrade;'

ssh -t -i -p 2294 buildbot@localhost \
  'sudo apt-get update && sudo apt-get -y dist-upgrade;'

copy the ttyS0 file to its proper location and shut down the VM

ssh -t -i -p 2293 buildbot@localhost \
  'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'

ssh -t -i -p 2294 buildbot@localhost \
  'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'

VMs for building .debs

for i in '/kvm/vms/vm-trusty-amd64-serial.qcow2 2293 qemu64' '/kvm/vms/vm-trusty-i386-serial.qcow2 2294 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
    "= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/thrift-0.9.0.tar.gz buildbot@localhost:/dev/shm/" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.5" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libpam0g-dev libreadline-gplv2-dev autoconf automake automake1.9 dpatch ghostscript-x libfontenc1 libjpeg62 libltdl-dev libltdl7 libmail-sendmail-perl libxfont1 lmodern texlive-latex-base-doc ttf-dejavu ttf-dejavu-extra libaio-dev xfonts-encodings xfonts-utils libxml2-dev unixodbc-dev bzr scons check libboost-all-dev openssl epm libjudy-dev libjemalloc-dev" \
    "bzr co --lightweight lp:mariadb-native-client" \
    "cd /usr/local/src;sudo tar zxf /dev/shm/thrift-0.9.0.tar.gz;pwd;ls" \
    "cd /usr/local/src/thrift-0.9.0;echo;pwd;sudo ./configure --prefix=/usr --enable-shared=no --enable-static=yes CXXFLAGS=-fPIC CFLAGS=-fPIC && echo && echo 'now making' && echo && sleep 3 && sudo make && echo && echo 'now installing' && echo && sleep 3 && sudo make install" ; \
done

VMs for install testing.

See Buildbot Setup for Virtual Machines - General Principles for how to obtain my.seed and sources.append.

for i in '/kvm/vms/vm-trusty-amd64-serial.qcow2 2293 qemu64' '/kvm/vms/vm-trusty-i386-serial.qcow2 2294 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils unixodbc libxml2 libjudydebian1 libjemalloc1" \
    "= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
    "sudo debconf-set-selections /tmp/my55.seed" \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
done

VMs for MySQL upgrade testing

for i in '/kvm/vms/vm-trusty-amd64-serial.qcow2 2293 qemu64' '/kvm/vms/vm-trusty-i386-serial.qcow2 2294 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils" \
    "= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
    "sudo debconf-set-selections /tmp/my55.seed" \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.5 libjemalloc1' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
done

VMs for MariaDB upgrade testing

for i in '/kvm/vms/vm-trusty-amd64-serial.qcow2 2293 qemu64' '/kvm/vms/vm-trusty-i386-serial.qcow2 2294 qemu64' ; do \
  set $i; \
  runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
    "= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
    "= scp -P $2 /kvm/vms/mariadb-trusty.list buildbot@localhost:/tmp/tmp.list" \
    "sudo debconf-set-selections /tmp/my55.seed" \
    'sudo mv -vi /tmp/tmp.list /etc/apt/sources.list.d/' \
    'sudo apt-key adv --recv-keys --keyserver pgp.mit.edu 0xcbcb082a1bb943db' \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mariadb-server libjemalloc1' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
    'sudo rm -v /etc/apt/sources.list.d/tmp.list' \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get update' \
    "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils' \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y'; \
done

Add Key to known_hosts

Do the following on each kvm host server (terrier, terrier2, i7, etc...) to add the VMs to known_hosts.

# trusty-amd64
cp -avi /kvm/vms/vm-trusty-amd64-install.qcow2 /kvm/vms/vm-trusty-amd64-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-trusty-amd64-test.qcow2 -redir tcp:2293::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2293 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-trusty-amd64-test.qcow2

# trusty-i386
cp -avi /kvm/vms/vm-trusty-i386-install.qcow2 /kvm/vms/vm-trusty-i386-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-trusty-i386-test.qcow2 -redir tcp:2294::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2294 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-trusty-i386-test.qcow2

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Ubuntu 8.04, 8.10, and 9.10

These 6 platforms were installed together (i386 and amd64).

Base installs

Ubuntu 8.04 "Hardy" amd64:

qemu-img create -f qcow2 vm-hardy-amd64-serial.qcow2 8G
kvm -m 1024 -hda vm-hardy-amd64-serial.qcow2 -cdrom /kvm/ubuntu-8.04.3-server-amd64.iso -redir tcp:2228::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-hardy-amd64-serial.qcow2 -cdrom /kvm/ubuntu-8.04.3-server-amd64.iso -redir tcp:2228::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2228 localhost
# edit /boot/grub/menu.lst and visudo, see below
sudo /sbin/shutdown -h now
kvm -m 1024 -hda vm-hardy-amd64-serial.qcow2 -cdrom /kvm/ubuntu-8.04.3-server-amd64.iso -redir tcp:2228::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -t -p 2228 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2228 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2228 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2228 ttyS0 buildbot@localhost:
ssh -p 2228 buildbot@localhost 'sudo cp ttyS0 /etc/event.d; rm ttyS0; sudo shutdown -h now'

Ubuntu 8.04 "Hardy" i386:

qemu-img create -f qcow2 vm-hardy-i386-serial.qcow2 8G
kvm -m 1024 -hda vm-hardy-i386-serial.qcow2 -cdrom /kvm/ubuntu-8.04.3-server-i386.iso -redir tcp:2229::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-hardy-i386-serial.qcow2 -cdrom /kvm/ubuntu-8.04.3-server-i386.iso -redir tcp:2229::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
ssh -p 2229 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2229 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2229 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2229 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2229 ttyS0 buildbot@localhost:
ssh -p 2229 buildbot@localhost 'sudo cp ttyS0 /etc/event.d; rm ttyS0; sudo shutdown -h now'

Ubuntu 8.10 "Intrepid" amd64:

qemu-img create -f qcow2 vm-intrepid-amd64-serial.qcow2 8G
kvm -m 1024 -hda vm-intrepid-amd64-serial.qcow2 -cdrom /kvm/ubuntu-8.10-server-amd64.iso -redir tcp:2230::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-intrepid-amd64-serial.qcow2 -cdrom /kvm/ubuntu-8.10-server-amd64.iso -redir tcp:2230::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2230 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2230 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2230 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2230 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2230 ttyS0 buildbot@localhost:
ssh -p 2230 buildbot@localhost 'sudo cp ttyS0 /etc/event.d; rm ttyS0; sudo shutdown -h now'

Ubuntu 8.10 "Intrepid" i386:

qemu-img create -f qcow2 vm-intrepid-i386-serial.qcow2 8G
kvm -m 1024 -hda vm-intrepid-i386-serial.qcow2 -cdrom /kvm/ubuntu-8.10-server-i386.iso -redir tcp:2231::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-intrepid-i386-serial.qcow2 -cdrom /kvm/ubuntu-8.10-server-i386.iso -redir tcp:2231::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
ssh -p 2231 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2231 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2231 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2231 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2231 ttyS0 buildbot@localhost:
ssh -p 2231 buildbot@localhost 'sudo cp ttyS0 /etc/event.d; rm ttyS0; sudo shutdown -h now'

Ubuntu 9.10 "Karmic" amd64:

qemu-img create -f qcow2 vm-karmic-amd64-serial.qcow2 8G
kvm -m 1024 -hda vm-karmic-amd64-serial.qcow2 -cdrom /kvm/ubuntu-9.10-server-amd64.iso -redir tcp:2232::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-karmic-amd64-serial.qcow2 -cdrom /kvm/ubuntu-9.10-server-amd64.iso -redir tcp:2232::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2232 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2232 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2232 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2232 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2232 ttyS0.conf buildbot@localhost:
ssh -p 2232 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

Ubuntu 9.10 "Karmic" i386:

qemu-img create -f qcow2 vm-karmic-i386-serial.qcow2 8G
kvm -m 1024 -hda vm-karmic-i386-serial.qcow2 -cdrom /kvm/ubuntu-9.10-server-i386.iso -redir tcp:2233::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-karmic-i386-serial.qcow2 -cdrom /kvm/ubuntu-9.10-server-i386.iso -redir tcp:2233::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
ssh -p 2233 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2233 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2233 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2233 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2233 ttyS0.conf buildbot@localhost:
ssh -p 2233 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

For visudo to enable passwordless sudo:

sudo VISUAL=vi visudo

# uncomment `%sudo ALL=NOPASSWD: ALL' line in `visudo`, and move to end.

For 8.04/8.10/9.04, /boot/grub/menu.lst is edited as follows:

sudo vi /boot/grub/menu.lst

# Add this:
  serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
  terminal --timeout=3 serial console

# Also add in menu.lst to kernel line (after removing `quiet splash'):
  console=tty0 console=ttyS0,115200n8

For 9.10, /boot/grub/menu.lst is edited as follows:

sudo vi /etc/default/grub

# Add/edit these entries:
    GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
    GRUB_TERMINAL="serial"
    GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"

sudo update-grub

VMs for building .debs

for i in 'vm-hardy-amd64-serial.qcow2 2228 qemu64' 'vm-hardy-i386-serial.qcow2 2229 qemu32,-nx' 'vm-intrepid-amd64-serial.qcow2 2230 qemu64' 'vm-intrepid-i386-serial.qcow2 2231 qemu32,-nx' ; do \
  set $i; \
  runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get build-dep -y mysql-server-5.0" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline5-dev" ; \
done

for i in 'vm-karmic-amd64-serial.qcow2 2232 qemu64' 'vm-karmic-i386-serial.qcow2 2233 qemu32,-nx' ; do \
  set $i; \
  runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.1" \
    "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline5-dev" ; \
done

VMs for install testing

See the General Principles article for how to make the 'my.seed' and 'sources.append' files.

for i in 'vm-hardy-amd64-serial.qcow2 2228 qemu64' 'vm-hardy-i386-serial.qcow2 2229 qemu32,-nx' 'vm-intrepid-amd64-serial.qcow2 2230 qemu64' 'vm-intrepid-i386-serial.qcow2 2231 qemu32,-nx' 'vm-karmic-amd64-serial.qcow2 2232 qemu64' 'vm-karmic-i386-serial.qcow2 2233 qemu32,-nx' ; do \
  set $i; \
  runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
	  "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
	  "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y debconf-utils" \
	  "= scp -P $2 my.seed sources.append buildbot@localhost:/tmp/" \
	  "sudo debconf-set-selections /tmp/my.seed" \
	  "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
done

VMs for upgrade testing

for i in 'vm-hardy-amd64-install.qcow2 2228 qemu64' 'vm-hardy-i386-install.qcow2 2229 qemu32,-nx' 'vm-intrepid-amd64-install.qcow2 2230 qemu64' 'vm-intrepid-i386-install.qcow2 2231 qemu32,-nx' ; do \
  set $i; \
  runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
  'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.0'
  'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ; \
done

for i in 'vm-karmic-amd64-install.qcow2 2232 qemu64' 'vm-karmic-i386-install.qcow2 2233 qemu32,-nx' ; do \
  set $i; \
  runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
    'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.1' \
    'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ; \
done

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Ubuntu 9.04 amd64

Base install

qemu-img create -f qcow2 vm-jaunty-amd64-serial.qcow2 8G
kvm -m 1024 -hda vm-jaunty-amd64-serial.qcow2 -cdrom /kvm/ubuntu-9.04-server-amd64.iso -redir tcp:2227::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

kvm -m 1024 -hda vm-jaunty-amd64-serial.qcow2 -cdrom /kvm/ubuntu-9.04-server-amd64.iso -redir tcp:2227::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user

sudo apt-get install emacs22-nox

Add to /boot/grub/menu.lst:

serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial console

also add in menu.lst to kernel line (after removing `quiet splash'):

console=tty0 console=ttyS0,115200n8

Create /etc/event.d/ttyS0:

# ttyS0 - getty
#
# This service maintains a getty on ttyS0 from the point the system is
# started until it is shut down again.

start on stopped rc2
start on stopped rc3
start on stopped rc4
start on stopped rc5

stop on runlevel 0
stop on runlevel 1
stop on runlevel 6

respawn
exec /sbin/getty 115200 ttyS0

Add account and allow passwordless sudo:

sudo adduser --disabled-password buildbot
sudo adduser buildbot sudo
sudo EDITOR=emacs visudo
# uncomment `%sudo ALL=NOPASSWD: ALL' line in `visudo`, and move to end.

sudo su -s /bin/bash - buildbot
mkdir .ssh
cat >.ssh/authorized_keys
# Paste public key for buildbot user on host system.
chmod -R go-rwx .ssh
# Do manual login from host to guest once, to make sure .ssh/known_hosts is updated.

VM for .deb building

qemu-img create -b vm-jaunty-amd64-serial.qcow2 -f qcow2 vm-jaunty-amd64-build.qcow2
kvm -m 1024 -hda vm-jaunty-amd64-build.qcow2 -cdrom /kvm/ubuntu-9.04-server-amd64.iso -redir tcp:2227::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

sudo apt-get build-dep mysql-server-5.1

VM for install testing

qemu-img create -b vm-jaunty-amd64-serial.qcow2 -f qcow2 vm-jaunty-amd64-install.qcow2
kvm -m 1024 -hda vm-jaunty-amd64-install.qcow2 -cdrom /kvm/ubuntu-9.04-server-amd64.iso -redir tcp:2227::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

sudo adduser --system --group mysql

_{This page is licensed: CC BY-SA / Gnu FDL}

Buildbot Setup for Virtual Machines - Ubuntu 9.04 i386

This vm is used to build source tarball and for Ubuntu 9.04 32-bit .deb.

First create and install image:

qemu-img create -f qcow2 vm-jaunty-i386-base.qcow2 8G
kvm -m 2047 -hda vm-jaunty-i386-base.qcow2 -cdrom ubuntu-9.04-server-i386.iso -redir tcp:2222::22 -boot d -cpu qemu32,-nx
# Install

Create a vm, based on the first one, which is configured for serial port.

qemu-img create -b vm-jaunty-i386-base.qcow2 -f qcow2 vm-jaunty-i386-serial.qcow2
kvm -m 2047 -hda vm-jaunty-i386-base.qcow2 -redir tcp:2222::22 -boot c -cpu qemu32,-nx

To configure kernel and grub for serial console, add the following to /boot/grub/menu.lst:

serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial console

also add in menu.lst to kernel line (after removing `quiet splash'):

console=tty0 console=ttyS0,115200n8

Create /etc/event.d/ttyS0:

# ttyS0 - getty
#
# This service maintains a getty on ttyS0 from the point the system is
# started until it is shut down again.

start on stopped rc2
start on stopped rc3
start on stopped rc4
start on stopped rc5

stop on runlevel 0
stop on runlevel 1
stop on runlevel 6

respawn
exec /sbin/getty 115200 ttyS0

Add an account:

sudo adduser --disabled-password buildbot

Copy public ssh key into /.ssh/authorized_keys

Enable passwordless sudo:

sudo adduser buildbot sudo
# uncomment `%sudo ALL=NOPASSWD: ALL' line in `visudo`, and move to end.

Create a new vm for building source tarballs:

qemu-img create -b vm-jaunty-i386-serial.qcow2 -f qcow2 vm-jaunty-i386-deb-tarbake.qcow2

Install compilers:

sudo apt-get build-dep mysql-5.1-server
sudo apt-get install devscripts hardening-wrapper bzr

Copy in an existing bzr shared repository to buildbot/.bzr (or run bzr init-repo and bzr branch --no-tree lp:maria).

Create a new vm for building .debs:

qemu-img create -b vm-jaunty-i386-serial.qcow2 -f qcow2 vm-jaunty-i386-deb-build.qcow2

Install compilers:

sudo apt-get build-dep mysql-5.1-server
sudo apt-get install devscripts hardening-wrapper

Create a new VM for testing .deb installs:

qemu-img create -b vm-jaunty-i386-serial.qcow2 -f qcow2 vm-jaunty-i386-deb-install.qcow2

Install tools and local apt repository.

sudo apt-get install debconf-utils
cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END

Setup default package config for debconf to enable unattended install. Copy in my.seed (see above) to vm.

sudo debconf-set-selections /tmp/my.seed

Create a new VM for testing upgrades:

qemu-img create -b vm-jaunty-i386-deb-install.qcow2 -f qcow2 vm-jaunty-i386-deb-upgrade.qcow2

Prepare initial MySQL install with some test data.

sudo apt-get install mysql-server-5.1
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"

_{This page is licensed: CC BY-SA / Gnu FDL}