Upcoming Oracle Virtualbox 3.2.x will introduce private namespace in OVF

Upcoming Oracle Virtualbox 3.2.x will introduce private namespace in OVF, this will allow to save/restore Virtualbox specific information in OVF files. See more upcoming changes in:

Oracle VirtualBox VirtualBox 3.2.0 Beta 1 released

Oracle introduce Oracle Virtual Assembly Builder for Virtual Appliances

“Oracle Virtual Assembly Builder is a new product designed to help organizations quickly and easily deploy multi-tier enterprise applications in Virtualized Environments. It also enables administrators to quickly configure and provision these applications.”

This is sound very interesting, reconfiguration of Applications is main challenge in Virtual Appliances.

Oracle announce new Virtualization Solutions

Criticism: On the many limitations of (network) virtual appliances

Interesting summary of issues with Virtual Appliances:

On the many limitations of (network) virtual appliances

Reducing OpenSolaris (Part1: Approaches and Strategies)

This entry is part of OpenSolaris JeOS (Just Enough Operating System) Prototype workshop.

Virtualization Assitant Live-CD PoC (Proof Of Concept)

Main page for Virtual Machines, Virtual Templates and Virtual Appliances Builder

Idea of building scalable builder is based on ideas and experience with simple builder service:

Virtual Appliance Build Service
Before full automation some more simple Proof of Concept project:

OpenSolaris 200906 Virtualization Assistant Live-CD Proof of Concept

Create a OpenSolaris 200906 Helper Live-CD Prototype with can help End Users and Developers to deliver faster VM Images/Templates/Appliances based on current mainline OpenSolaris 200906 release.
VA Live-CD will as PoC (Proof of Concept) demonstrate all known and needed steps of VM creation as modules or/and their recipes taking as a sample building process of OpenSolaris 200906 JeOS VM install and creation itself.

Content

1. OpenSolaris 200906 Virtualization Assistant Live-CD PoC
2. Project Motivations
3. Target Audience of OpenSolaris 200906 Virtualization Assistant Live-CD Prototype
4. Planned Functionality of OpenSolaris 200906 Virtualization Assistant Live-CD PoC
   1. Virtualization Assistant Live-CD PoC Deliverables: Media and Repository
      1. Virtualization Assistant Live-CD PoC Media Download
      2. Virtualization Assistant Live-CD PoC Mercurial repository
   2. VA Live-CD Functionality – Separation onto 2 sub-projects
   3. Content
   4. OpenSolaris 200806 Migration Assistant Live-CD
   5. OpenSolaris 200906 VA Live-CD Main Design Concepts
      1. AI Client in Live Media Based Rich Recovery Console
      2. Wrapping Functionality in “Modules”
      3. Modules are using Critical Check Points and Acummulated Return Values
      4. Overlays for delivering VA Live-CD, AI and Install framework Fast Fixes
      5. Include DC and all Build Recipe on Live-CD Media Itself
      6. Only Virtualization Platforms and x86 32bit support (and SPARC LDoms)
      7. Maximally Leverage OpenSolaris 200906 Distro Constructor
      8. Include Headless Mode with SERIAL port access (OS Kernel and Grub Menu)
      9. Include SSH key based authentication for better scripting
      10. Support for Alternate Install Scenarios: Fully Local Installation
      11. Support for Alternate Install Scenarios: ZFS Flash like install
      12. Setting up JeOS logins (not default OpenSolaris ones)
   6. Part 1: Building VA Live-CD Media with Distro Constructor
      1. VA Live-CD Media Distro Constructor Recipe
      2. VA Live-CD Media Distro Constructor Helper Files
      3. Customize VA Live-CD by Generating Clone from Live-CD
         1. Sample 1: Regeneration of VA Live-CD with Customizations (lofs)
         2. Sample 2: Generation of VA Live-CD from itself with added WiFi Intel …
      4. VA Live-CD media Costs and Sizes (Architectures Comparisons)
   7. Part 2: Virtual Assistant Live-CD Modules for VM Builders
      1. What are VA Live-CD modules and where they come from ?
      2. Module for testing main VA Live-CD functionality
         1. Selecting Best Strategy for Repeatable AI based JeOS Installations
      3. Module with common functionality
      4. Module for Local IPS Repository Recipe
      5. Module for Installing OpenSolaris 200906 JeOS
      6. Module for cleaning OS Runtime Data
      7. Module for HW arch change and HW Reconfiguration
      8. Module for securing installed OS instance
      9. Module shrinking disk with ZFS send/receive with 2 disks
      10. Module shrinking disk with ZFS send/receive with archive NFS
      11. Module for Virtual Hardware support
   8. VA Live-CD and JeOS Support for 10+ Most Popular Virtualization Platforms
      1. List of Know to Work Virtualization Platforms (x86 , x86-64)
      2. List of Know to Work Virtualization Platforms (SPARC SUN4V)
      3. HW Arch Check: xVM Hypervisor Para mode
      4. HW FAST FIX: VGIRUni network driver for Parallels
         1. Create local IPS repository on port 80
         2. Publish LOCALni into IPS repository
      5. HW FAST FIX: Update Dnet driver from B113
   9. Creating “instance” zpool for Live-CD custom data

OpenSolaris JeOS (Main Page)

Main page for OpenSolaris JeOS (Just Enough Operating System) Prototypes related stuff.

News: Lightning Talk about OpenSolaris JeOS at Community One (1-Jun-09 2:40PM)

General Information – JeOS Prototype OpenSolaris project

   JeOS: Just enough OS Project Homepage

   OpenSolaris 2009.06 JeOS Prototype VM Images Download

   Discusion jeos-discuss@opensolaris.org (Main page)
   Discusion jeos-discuss@opensolaris.org (Archives)

10 Simple JeOS Prototype Usage Samples (Based on JeOS VM images)

I prepare in this blog 10 simple JeOS Prototype Usage Samples for your inspiration:

1. Security related Enhancements in OpenSolaris JeOS Prototype
2. JeOS like Zones in OpenSolaris JeOS Prototype
3. NAS storage server in OpenSolaris JeOS Prototype
4. C compile enviroment in OpenSolaris JeOS Prototype
5. Personal SourceJuicer in OpenSolaris JeOS Prototype
6. WebStack (Apache,MySQL,PHP) in OpenSolaris JeOS Prototype
7. WebStack (Tomcat, MySQL, Java) in OpenSolaris JeOS Prototype
8. Glassfish (Glasfish v2, MySQL, Java) in OpenSolaris JeOS Prototype
9. Ruby Web Dev Env (Ruby Rack, MySQL, Ruby Rails) in OpenSolaris JeOS Prototype
10. Minimal X env for iPKG GUI in OpenSolaris JeOS Prototype ^*
    1. Part 1: Minimal X env for iPKG GUI in OpenSolaris JeOS Prototype
    2. Part 2: Minimal X env for iPKG GUI in OpenSolaris JeOS Prototype
    3. Part 3: Minimal X env for iPKG GUI in OpenSolaris JeOS Prototype

 * Note: Adding minimal X environment will install in CLI JeOS Prototype a lot of GUI based components, but GUI functionality will be still limited to certain simple use cases.

General Information – Overview

• OpenSolaris JeOS Prototype (Motivations)
• OpenSolaris JeOS Prototype Deliverables
• OpenSolaris JeOS Prototype Motivations
• OpenSolaris JeOS Prototype Roots
• OpenSolaris JeOS Prototype (Design Basis)
• How we design and create OpenSolaris JeOS Prototype
• OpenSolaris JeOS Prototype Design Flexibility
• What Works Well
• Most Visible Challenges
• Main Outcome

Developers Corner – Implemetation

• OpenSolaris JeOS Prototype (Audience)
• Target audience of this JeOS Prototype
• OpenSolaris JeOS Prototype (Linux vs OpenSolaris)
• OpenSolaris design limitations and JeOS construction possibilities
• Linux
• Common Linux JeOS Implementaions
• OpenSolaris
• OpenSolaris JeOS Prototype (Reduction vs Minimalization)
• Reduction strategy (Scheme) applied to OpenSolaris 2008.11 / 2009.06 to get JeOS Prototype
• Multiple Reduced OpenSolaris Setups with BE feature
• Main Idea – Use different Boot Environments in single image
• Multiple Boot Environments
• Create Primary OpenSolaris 2000906 Boot Environment (BE)
• Create JeOS configs for various OpenSolaris 200906 Boot Environment (BE)
• L0 osol0060gui Backup of Original Installation with GUI Boot mode
• L1 osol0906cli Original Installation with CLI Boot mode
• L2 osol0906size Remove Most Top Sized User Packages
• L3 osol0906rgui Remove main GUI front end part
• L5 osol0906redu Try JeOS Uninstall Rest scenario
• L5 osol0906inst Try JeOS Install List scenario
• Removing original GUI BE, leaving just customized ones
• Install Base Targets applied to OpenSolaris 2008.11 / 2009.06
• JeOS Installation costs
• JeOS Execution costs
• JeOS Install Cost Deep Dive
• JeOS Install packages lists
• JeOS Install per packages cots
• OpenSolaris 2010.03 JeOS Protoype First Touch on Development Branch
• OpenSolaris 1003 Dev B130 JeOS Prototype First Touch (Part One)

   Approach to OSOL 1003 Dev B130 JeOS First Touch
   Installing OpenSolairs 201003 B130 Dev Virtual Environment
   Initial Evaluation of changes in B130
      Initial install evaluation of main 3 packages
      Check if issues with X and Gnome stuff is fixed for SUNWrmvolmgr and SUNWgnu-mc
      Check if some useful stuff from CONTRIB was moved to dev (release)
      Check if migration to Python 2.6 was completed on B130
   B130 JeOS Protototype First Touch pkgs lists and costs
      Install costs for B130 JeOS Protototype First Touch pkgs lists
      Try to generate AI manifest body

• OpenSolaris 1003 Dev B130 JeOS Prototype First Touch (Part Two)

   B130 JeOS Protototype First Touch AI installation
      Create B130 JeOS Prototype First Touch AI manifest
      Install B130 JeOS Prototype First Touch AI manifest with AI on Media
      Evaluate AI installation of B130 JeOS Prototype First Touch
      Generate full pkgs simple list
      Generate full pkgs advanced list
      Evaluate install and distributions costs

• OpenSolaris 1003 Dev B130 JeOS Prototype First Touch (Part Three)

   iPKG GUI tools on OSOL 1003 JeOS CLI Firts Touch
      Evaluating iPKG GUI in OSOL 1003 B130 JeOS First Touch
      Installing iPKG GUI in OSOL 1003 B130 JeOS First Touch
      Trying VNC session based iPKG GUI in OSOL 1003 B130 JeOS First Touch
      AI install hints for iPKG GUI in OSOL 1003 B130 JeOS First Touch
      Install costs for iPKG GUI in OSOL 1003 B130 JeOS First Touch

• OpenSolaris 1003 Dev B130 JeOS Prototype First Touch (Part Four)

   OSOL 1003 JeOS CLI First Touch on Real HW
      Hardware Appliance OpenSolaris JeOS like Sample
      Get list of drivers and its packages to add to OpenSolaris JeOS
      Install OpenSolaris JeOS Prototype with added drivers
         A: Modify JeOS AI install manifest to install drivers in one shot
         B: Install default JeOS AI install manifest and add drivers to installation before reboot

• OpenSolaris 1003 Dev B130 JeOS Prototype First Touch (Part Five)

   OpenSolaris JeOS Prototype – Reported Issues (Bugs) and Suggestions (RFEs)
      OSOL1003 reported issues (B130 and newer)
      OSOL1003 OS Related Features

      Reported Issues during OSOL JeOS Prototyping (OSOL 200805 – 200906)

OVF Standard Overview (Main Page)

OVF (Open Virtualization Format) Standard Overview

In this Corner I will try to consolidate our knowledge and views to OVF (Open Virtualization Format), new Virtualization standard which promise to deliver cross-platform vendor independent distribution framework

• OVF (Open Virtualization Format) Standard Origins
• OVF Standard Sources
• Reference Implementation Open-OVF
• SW with OVF support
• OVF Tool implementations
• OVF Import implementations
• OVF Export implementations

OVF (Open Virtualization Format) Standard Overview (Part 1)

In this Corner I will try to consolidate our knowledge and views to OVF (Open Virtualization Format), new Virtualization standard which promise to deliver cross-platform vendor independent distribution framework

OVF (Open Virtualization Format) Standard Origins

OVF (Open Virtualization Format) is framework originally proposed by VMware & XenSource with contributions from Dell, HP, IBM and Microsoft as neutral standard , later it was accepted as DMTF standard (Released on 23/Mar/2009).

While many public comment then OVF define simple portable Virtual Machines, this is common misinterpretation of OVF, because it try to address address wider area of Virtual Appliances and is part of wider standardization effort called “Virtualization Management (VMAN) Initiative”

OVF Standard Sources

• VMware Sources
  • Man Page: What is OVF?
  • Open Virtualization Format Whitepaper (Older – Larger One)

• DMTF Standard Pages
  • Enabling Portability & Simplified Deployment of Virtual Appliances
  • Open Virtualization Format (OVF) Version 1.0 Published as a DMTF Standard.
  • Virtualization Management (VMAN) Initiative Homepage (OVF amd CIM standards)

• Other Sources
  • Industry players working on standard for VMs (1997)
  • Major Server Builders Opt For a Common Virtualization Standard (They target the “zero configuration virtualization” concept) (1997)
  • Microsoft to Deliver a Platform Agnostic Standard for Virtualization (1997)
  • Open Virtualization Format and virtual appliances (2008)
  • Open Virtual Machine Format to benefit users (2008)
  • A review of (Open Virtual Machine Format) OVF from a systems management perspective (2008)
  • Reviewing DMTF OVF as a “preliminary standard” (2008)
  • OVF in action: Kensho

Reference Implementation Open-OVF

Sometimes OVF Standard and related docs like CMI profiles is difficult to follow up, I use a Open-OVF Open Sourced implementation everywhere I need to get deep implementation ideas

• Open OVF Proposal
• Open-OVF Homepage
• Open-OVF Command Line Interface

Open-OVF is implemented in Python, is developed on/for Linux , but must be easily portable to (Open)Solaris too.

Open-OVF is integrated with other Open Source Virtualization tools like virtlib

SW with OVF support

Most referred SW here was implemented before official OVF 1.0 standard was published.

OVF Tool implementations

• VMware vCenter Converter (Version 3.x is for Windows only, Version 4.x also support Linux)
• VMware OVF tool (Originaly it was Java now it is a Linux/MS Windows tool ?)
• VMware Studio Virtual Appliances creation tool from VMware
• Open-OVF (Don’t solve Virtual disk conversions)
• IBM’s Open Virtualization Format Toolkit (Need Eclipse)
• Citrix Kensho Project
• Others (Add here)

Note: I often us use Open-OVF as reference implementation.

OVF Import implementations

• VMware ESX 3.5u3 and ESXi 3.5u3 support OVF
• All latest VMware Desktop Versions support OVF and OVA, local and net based
• Sun xVM VirtualBox 2.2.x (OVF only) (Beta 1 , Beta 2 and Pre-RR builds for now)
• Others (add there)

OVF Export implementations

• VMware Workstation 6.5.x (Have Converter Client)
• Sun xVM VirtualBox 2.2.x (Beta 1 for now)

Demonstrating VirtualBox 2.2 providing VMware Wrk 5.x + ESX compatility

For OpenSolaris Virtual Image and Virtual Appliances creation process most important is , if we can load VMDK file created on VirtualBox into VMware Workstation in ESX server family products.
For example for Devel, QA and LAB services can be important to make direct VMDK disk migration without to go throw painful OVF virtual disk compression/decompression.

This entry is part of Mastering Sun VirtualBox Workshop

Home No-Network needed OpenSolaris Live-CD IPS REPO full mirror

One time my child was sick I need to work from home for longer time, unfortunately I was in this time assigned to project required a IPS repository access.
My network a home is not enough stable to use IPS remote mirror, so I as workaround use this script to create:

I think you can find this approach and script helpful in home Virtual Appliances prototyping.

Virtual Appliances & OVF Workshop

Main page for our expertize about Virtual Appliances & OVF
Here I try to present some practical information from Virtual Appliances creation processes used during this related projects:
Virtual Appliances

My colleague, Gabor create a series of blog entries about high level 7 steps Virtual Templates creation process:

• VM template assembly process at glance
• VM template assembly details: Step 1 – Create JeOS Instance
• VM template assembly details: Step 2/3 – Add layered SW and template internals
• VM template assembly details: Step 4/5 – Clean-up and finalize the template
• VM template assembly details: Step 6/7 – Convert and compress the VM

I start working on small real sample of Virtual Appliance / VM template creation using 7 steps process:

Glassfish V3 Pet Catalog sample DEMO in VM Template

• Part 1: Glassfish V3 Pet Catalog sample DEMO in VM Template – Preparation

Preparation: Create basic Glassfish V3 VM Template outline
   1. Learn more about VM Templates creation process
   2. Plan carefully VM Template content and audience
   3. Check if all panned to use products are FOSS based

• Part 2: Glassfish V3 Pet Catalog sample DEMO in VM Template – OS Instance

Step 1: Create OS Instance
   General Information – JeOS Prototype OpenSolaris project
   Step 1a: Create an OpenSolaris 20906 JeOS Prototype instance
   Step 1b: Start JeOS VM instance and check readiness as user root
   Step 1c: Create a non-root but fully privileged user

• Part 3: Glassfish V3 Pet Catalog sample DEMO in VM Template – SW Installation

Step 2: Add layered software
   Step 2a: Bundled vs UnBundled SW installation
   Step 2b: Using UnBundled SW Installation for our DEMO
   Step 2c: OpenSolaris is dual arch (32bit and 64bit) environment
   Step 2d: Creating a dedicated ZFS fs for UnBundled installation

• Part 4: Glassfish V3 Pet Catalog sample DEMO in VM Template – SW Installation

   Step 2e: Install and Configure Glassfish V3 (Community Edition)

• Part 5: Glassfish V3 Pet Catalog sample DEMO in VM Template – SW Installation

   Step 2f: Install and configure MySQL (WebStack)

• Part 6: Glassfish V3 Pet Catalog sample DEMO in VM Template – SW Installation

   Step 2g: Install Java EE 6 Pet Catalog sample from NetBeans 6.8 IDE (Video)

• Part 7: Glassfish V3 Pet Catalog sample DEMO in VM Template – Adding internals

Step 3: Add internals
   Step 3a: Add auto starting of layered components
   Step 3b: Adding welcome WEB page
   Step 3c: Licenses audit
  Final check for usage size after ALL SW installation

• Part 8: Glassfish V3 Pet Catalog sample DEMO in VM Template – Cleaning

Step 4: Clean-up the template
      Step 4a: Motivations for Clean-up
      Step 4b: Clear layered app SW
      Step 4c: Clean OS and default user OSOL data
      Step 4d: Important: Time to make a backup now
      Step 4e: Shrink installed system disk with ZFS Streaming
            First I will optimize disk with local ZFS SEND/RECEIVE streaming
            Second I will optimize disk with NFS based ZFS SEND/RECEIVE streaming

• Part 9: Glassfish V3 Pet Catalog sample DEMO in VM Template – Finalize

Step 5: Finalize the template
      Step 5a: Finalize template outline

• Part 10: Glassfish V3 Pet Catalog sample DEMO in VM Template – Clean and Convert

Step 6: Clean and Convert VM
      Step 6a: Clean VMware – our primary development platform
      Step 6b: Generate VirtualBOX 3.x OVF format

• Part 12: Glassfish V3 Pet Catalog sample DEMO in VM Template – Clean and Convert

Step 6: Clean and Convert VM
   Step 6c: Generate VirtualBOX 3.x VDI IDE format
      Convert VMDK SCSI to VDI IDE format with VirtualBox
      Configure VirtualBox for VDI IDE
      Reconfigure HW with VA Live-CD in VirtualBox?for VDI IDE
   Step 6d: Generate Parallels format

      Open in Parallels JeOS image and recreate OS disk
      Restore from ZFS stream from VA Live-CD
      Add Freeware NI network driver
   Step 6e: Generate xVM/XEN compatible PARA and HVM RAW disk formats



• Part 11: Glassfish V3 Pet Catalog sample DEMO in VM Template – Pack and Publish

Step 7: Pack and Publish Images
      Step 7a: Pack each Image with compression tool
      Step 7b: Generate a checksum for each archive
      Step 7d: Create a Download page with helper instructions

Download: Glassfish V3 Pet Catalog sample DEMO in VM Template

Some interesting related to Virtual Alliances / VM Tempaltes links

• Compressed ZFS Stream sizes in various compressing programs and OVF
• VirtualBox 2.2 provide VMware Wrk 5.x + ESX compatible devices