Setting up Linux on Windows with VirtualBox

 

After trying and failing to run Ubuntu as my primary OS (for reasons I am not in the mood to discuss, I relented and installed Windows 7 Professional. (I bought a heavily discounted copy through Friends and Family program at the Microsoft company store. Never pay retail price.)
 
I use VirtualBox to host a virtual Linux machine inside my Windows Desktop. This setup lets me to have a (vaguely) usable Desktop console experience, with sleep/wake and power management, wireless, stable desktop graphics, and all those nice things.

This article describes how I set up VirtualBox to host Ubuntu as a Guest OS running inside Windows. A separate article will discuss how I set up the purely-Linux bits. This article discusses how to install and configure Virtual Box, and how configure the Linux<->Windows connection.

Install VirtualBox and create a virtual machine.

Install VirtualBox 4.1.2

Create a new Virtual Machine, following the prompts.

I like to separate my large “special” files from the rest of my home directory. This makes backups easier – I can frequently backup my small set of personal files, and backup large media/system files on a different schedule.

So, change the default locations for the Virtual Disk Image (VDI) and VM Snapshot, since these are very large files. I use C:\data.

You’ll need to install an OS on your virtual machine. I use Ubuntu. In VM Settings –> Storage –> IDE Controller, click on the CD icon on the right and choose a virtual CD/DVD (ISO image file) to “insert” into the virtual optical disk drive.

Start your VM, and it will boot from that “virtual CD” and let you install an operating system (called the Guest OS on your PC, distinguished from the Host OS, which is Windows).
Once the Guest OS is installed, I strongly recommend that you install an SSH server as soon as possible, so that you can connect to your virtual machine without using the virtual display, which is memory-intensive and CPU-intensive.

Guest Extensions

Guest Extensions is a package of software that runs inside the Guest OS, to enable special Virtual Box features. Generally, these are “virtual drivers” that let the Guest OS communicate with VirtualBox, enabling great features like fancier mouse/keyboard support, higher/variable “virtual display” resolution, and exposing host system directories as virtual hard drive partitions in the guest.

After installing and booting the OS, install “Guest Extensions”. To do this, pick the option from the Menu at the top of the window in which the Guest OS is running. That will mount a virtual CD. Run the “autorun.sh” shell script (as root) from that virtual CD, and Guest Extensions will install. Guest Extensions fixes GUI resolution, making the Guest OS display size match the window size, which you can change by clicking and dragging on the corner of the window.

After installing Guest Extensions, shut down the Guest OS, so we can finish setting it up.

Extension Pack

Extension Pack is different from Guest Extensions.  Extension Pack is a package of Host software that extends the functionality of VirtualBox. Extension Pack has the non-Open Source code of Virtual Box, which is why it is packaged as an extension.

The most important Extension Pack feature is USB 2.0 filter support. This feature lets your guest OS interact directly with your Host machine’s USB 2.0 devices. Download and install Extension Pack, and then visit Settings –> USB in VirtualBox Manager. Here you can check the Enable USB 2.0 box, and then click on the “USB-plus” icon to choose devices you want to provide to the Guest OS.

One very important USB 2.0 device is our external hard drive, which has a Linux formatted (ext-format) partition that Windows cannot see.

Networking

VirtualBox uses NAT (network address translation) networking by default, which is (probably) the same setup that your home PC uses to connect to the Internet. This is a one-way sort of networking:  It’s easy to reach the Internet from the machine, but hard to reach the machine from the Internet. In the case of a virtual machine, your local network is considered part of the Internet. This is probably not want you want.

Most likely, you’ll want to connect to the Guest OS via ssh, or a webserver, or some other network protocols, besides a direct virtual console (keyboard/mouse/display) connection. To do this, you’ll need to set up either Bridged Networking or  Port Forwarding.

If you want to treat your VM like a real machine on your local network, so that you connect to services (SSH, HTTP) running on the machine in the “usual” way,  then you’ll want to set up Bridged Networking, which basically instructs Virtual Box to run a virtual switch to direct traffic between the virtual machine (one one side) and the Host machine and rest of the local network network (on the other side)

Bridged Networking

While our virtual machine is powered off, open VirtualBox Manager and navigate to Settings –> Network –> Adapter 1. Change the “Attached to:” option from NAT to Bridged Adapter. Make sure Cable Connected is checked.

Now you are set up for full bidirectional network connectivity on your virtual machine. Have fun!
Also, beware: Your virtual machine is now equally exposed to the Internet as your host machine, so be conscious of security.

In particular, if you follow the (very wise) practice of running a web browser inside a virtual machine, in order to isolate your PC from malicious web sites, then you need to turn off Bridged Networking on any virtual machine that you want to quarantine from the Internet.

Port forwarding

Skip this section if you have chosen Bridged Networking.

I wrote a simple shell script (requires Cygwin to run) to set up Port Forwarding from the Windows desktop to the VirtualBox guest Linux system. This script sets up forwarding from Windows:2222 –> Linux:22 for ssh, Windows:8787 –> Linux:8787for Rstudio, and 8000 –> 80 for http. I hope it’s obvious how you can modify it to set up the services/ports you need.

I named this script virtualbox-port-setup. Run it like so:
  virtualbox-port-setup VM_Name

#!/bin/bash

test "$1" == "" && echo "USAGE: $0 guest_name" && exit 1
guestname=$1
net_path="VBoxInternal/Devices/pcnet/0/LUN#0/Config"
function run() {
    echo "$@"
    "$@"
}
function forward_port() {
    cmd="VBoxManage setextradata $guestname"
    local service=$1
    local host_port=$2
    local guest_port=$3
    local protocol=$4
    
    run $cmd "$net_path/$service/HostPort"  $host_port
    run $cmd "$net_path/$service/GuestPort"  $guest_port
    run $cmd "$net_path/$service/Protocol" $protocol

}
case $2 in
    undo)
      # unset settings
    forward_port ssh 
    forward_port rstudio 
    ;;
    *)

    forward_port ssh 2222 22 TCP
    forward_port rstudio 8787 8787 TCP
    forward_port http 8000 80 TCP
    ;;
esac
    
echo
echo "Updated NAT config:"
run VBoxManage getextradata $guestname enumerate | grep $net_path

(Side note: I am using Code Formatter Plugin for Windows Live Writer to insert code snippets into this article.)

You can run this script at any time, but you’ll need to completely close down your Guest OS (I think saving state is fine) in order to complete the setup and activate port forwarding.

Important! Activating port forwarding breaks your virtual machine. (There is a fix.) After closing the virtual machine, open VirtualBox Manager, edit Settings for the VM, navigate to the “Network” section, and change “Adapter Type” from Intel to PCnet-FAST III Am79C973. This change will fix the error “Configuration error: Failed to get the "MAC" value. VBox status code: -2103 (VERR_CFGM_VALUE_NOT_FOUND).”



If you ever decide to remove the NAT, run this script with the “undo” argument, (or else your virtual machine will fail to start up, giving an error about “unknown configuration node”):

   virtualbox-port-setup VM_Name undo

Running Headless

You could launch the  virtual machine now, but here’s a better idea: Run the VM headless. Headless VM has several benefits:

1. You can save the memory and CPU cost of running a graphical desktop environment inside the Guest OS.
2. You can use a host-side (Windows) X Server that will let you run each of your graphical guest/Linux programs “rootless” on the Windows desktop, instead of all locked inside a single Linux container window.

To manage your headless guest OS, use these commands:

# start
"C:\Program Files\Oracle\VirtualBox\VBoxHeadless.exe" --comment "<VM name>" --startvm "00000000-0000-0000-0000-000000000000"

# suspend
"C:\Program Files\Oracle\VirtualBox\VBoxManage.exe" controlvm  "<VM name>" savestate

You’ll need to change the <VM NAME> and GUID (long string of digits) to match your VM. An easy way to get this information is to enter VirtualBox Manager, right-click on your VM entry, and choose “Create a Shortcut on Desktop” Then right-click on the shortcut, choose Properties, and copy-paste the VM specs.

 

Connecting your Headless VM

Remember when I told you to install an ssh server? I hope you followed my advice. We are going to connect to the virtual machine over ssh, and forward graphics and sound back to the local (Host) machine.

Before we connect, we need to prepare the Host system to play audio and video from the guest system.

Simple: Audio Setup: Use your head

The simplest way to play sound is through the “Head” console.

To do this, start your virtual machine in the “normal” way (non-headless), and login to the desktop session.

Then, any programs you run (on the head console or on a headless connection) will send audio to the head console for VirtualBox to play for you.

The idea here is that you can still run rootless GUI windows on your Host system, while playing audio through VirtualBox.

Running the head console loses some of the RAM-savings and CPU-savings of pure headless mode, so you’ll want to configure your virtual machine’s desktop session to be a minimalist lightweight desktop that can still play audio.

If you want to run totally headless, follow the advice for “Headless Linux” in the Linux article in this series.

Now we are ready to connect!

If you are using NAT Port-Forwarding: To connect to your VM, ssh to the forwarded port on the host machine, using the username and password of an account on the guest OS.

If you are using Bridged Networking: To connect to your VM, ssh to the guest machine hostname/IP, using the username and password of an account on the guest OS.

• Add the –Y option to enable X forwarding, so you can see the GUI of guest OS programs on your local desktop.
• Add the –R option to set up a tunnel for eSound

Port-forwarded NAT networking:

    ssh –Y -C -R 33001:localhost:16001  guest_os_user@host_machine –p 2022

Bridged networking:  

    ssh –Y -C -R 33001:localhost:16001  guest_os_user@guest_machine

(This works using Cygwin/POSIX. You might need to tweak things to run from a Windows command shell.)

Raw hard-drive access

I have a Linux-formatter hard disk from my machine’s briefly previous life running Linux. I would like to access that disk directly from Linux. Here is how we do it:

WARNING: Do not attempt to mount any disk/partition that is also mounted by the Host system, especially not the Host boot disk!

WARNING: Some programs, like Nautilus, will auto-detect all partitions on a hard drive, and attempt to mount them if you click on an icon. Clicking the icon for any partition mounted by the Host OS will crash the Guest OS!

Create VMDK files. Here is a Cygwin/bash script that can help with that.

drive_infos=$@for drive_info in $drive_infos ; do
        drive_number=$(echo -n "$drive_info" | sed -e 's/:.*$//')        partitions=$(echo -n "$drive_info" | sed -e 's/.*://')        if [ "$partitions" == "" ] ; then                partitions_flag=""        else                partitions_flag="-partitions $partitions"        fi        vmdk_file=$vmdk_dir/RawDisk-PhysicalDrive${drive_number}-$partitions.vmdk        echo_run VBoxManage internalcommands createrawvmdk -filename $vmdk_file $partitions_flag -rawdisk '\\.\PhysicalDrive'$drive_number

In VirtualBox Manager, Settings –> Storage, add the newly VMDK files as virtual disks. (Check “Solid State” if appropriate.)



Use  “listpartitions” to find the partitions numbers on your disks, if you want to only expose non-Windows partitions to Linux. Then you can pass drive numbers like “1:3,4” to the script above, for drive 1, partitions 3 and 4):

VBoxManage.exe internalcommands  listpartitions -rawdisk '\\.\PhysicalDrive0'

VBoxManage.exe internalcommands  listpartitions -rawdisk '\\.\PhysicalDrive1'

VBoxManage.exe internalcommands  listpartitions -rawdisk '\\.\PhysicalDrive2'

Guest OS setup

Login to the Guest OS for the rest of this section.

Create mount-point in Guest OS, and set its group to vboxsf, like other shared folder mountpoints:

   mkdir /media/internal-storage-drive

   chgrp vboxsf /media/internal-storage-drive

Run blkid to find the UUIDS for all the drives of interest, which we will need shortly: (Credit: liquidat)

  blkid /dev/sd*

Create entry in /etc/fstab in Guest. Set options appropriately based on Filesystem type. Study this example, and then write your own version to suit your system:

##### /dev/sdb – Internal storage drive

#/dev/sdb1 – Internal SSD

UUID=4957fb72-... /media/internal-storage-drive ext4    user,group,rw              0       2

#/dev/sdb2 – Swap partition we’ll ignore for now.

##### /dev/sdc: External USB drive

# /dev/sdc1: MBR or something

# /dev/sdc1: LABEL="EFI" UUID="70D6-1701" TYPE="vfat"

#noauto, at least until we install hfsplus drivers and everything looks stable
# /dev/sdc2: UUID="aa810eb7-4048-3b0f-9ecf-8a2d6841337b" LABEL="Elements-Mac" TYPE="hfsplus"
LABEL=Elements-Mac /media/external-elements-mac  hfsplus    rw,noauto              0       2

# NOT SAFE to mount ntfs drive that is also mounted by Host OS!
#/dev/sdc3: LABEL="Elements-NT" UUID="52FB51860ACFBBED" TYPE="ntfs"
#LABEL=Elements-NT  /media/external-elements-nt  ntfs    user,group,rw              0       2

# /dev/sdc4: LABEL="Linux" UUID="59a3928e-d38f-4855-837a-31d32562fac9" TYPE="ext4"
UUID=59a3928e-...   /media/external-elements-linux  ext4    user,group,rw              0       2

# /dev/sdb5: UUID="08045a62-85bc-4028-b23e-c05670d6cca0" TYPE="swap"

Make your user a member of “vboxsf” group that can mount modify mountpoint. (vbosxf is overloaded – I’m using the Shared Folder group for Host disks)

sudo usermod –append –groups vboxsf

in gparted in Guest, remove “Boot” flag (for safety, for now)

sudo gparted

Mount!

 mount /dev/sdb1    or    mount -a

Note: If the userids on the Guest OS do not match the user ids on the hard drive, you’ll have permissions problems. If you have problems, you’ll need to re-number users on the Guest OS (consult OS documentation, /etc/passwd), or change owners of files on the hard drive (chown –R).

WARNING: Do not attempt to mount any disk/partition that is also mounted by the Host system, especially not the Host boot disk!

WARNING: Some programs, like Nautilus, will auto-detect all partitions on a hard drive, and attempt to mount them if you click on an icon. Clicking the icon for any partition mounted by the Host OS will crash the Guest OS!

References: http://www.sysprobs.com/access-physical-disk-virtualbox-desktop-virtualization-software

USB devices

I thought I needed to set this up, but I was able to access my external USB hard drive using the “raw hard drive access” method above.

Audio

Should work automatically. Audio works for me when I connect to the “head” console of the VM (non-headless mode). When I connect to the VM over ssh and run a program there, sound does not transmit back to my client.

This is a general challenge of running a program on a remote machine, not VirtualBox specific.

To Be Continued…

There is a lot more Linux setup to do, but it’s not VirtualBox specific. We’ll cover that material in another article.