Oracle VM Server for SPARC

Logical Domains (LDoms or LDOM) is the server virtualization and partitioning technology for SPARC V9 processors. It was first released by Sun Microsystems in April 2007. After the Oracle acquisition of Sun in January 2010, the product has been re-branded as Oracle VM Server for SPARC from version 2.0 onwards.

Each domain is a full virtual machine with a reconfigurable subset of hardware resources. Domains can be securely live migrated between servers while running. Operating systems running inside Logical Domains can be started, stopped, and rebooted independently. A running domain can be dynamically reconfigured to add or remove CPUs, RAM, or I/O devices without requiring a reboot. Using Dynamic Resource Management, CPU resources can be automatically reconfigured as needed.^[2]

SPARC hypervisors run in hyperprivileged execution mode, which was introduced in the sun4v architecture. The sun4v processors released as of October 2015 are the UltraSPARC T1, T2, T2+, T3,^[3] T4,^[4] T5, M5, M6, M10, and M7. Systems based on UltraSPARC T1 support only Logical Domains versions 1.0-1.2.^[5] The newer types of T-series servers support both older Logical Domains and newer Oracle VM Server for SPARC product version 2.0 and later. These include:

Technically, the virtualization product consists of two interdependent components: the hypervisor in the SPARC server firmware and the Logical Domains Manager software installed on the Solaris operating system running within the control domain (see Logical Domain roles, below). Because of this, each particular version of Logical Domains (Oracle VM Server for SPARC) software requires a certain minimum version of the hypervisor to be installed into the server firmware.

Logical Domains exploits the chip multithreading (CMT) nature of the "CoolThreads" processors. A single chip contains up to 32 CPU cores, and each core has either four hardware threads (for the UltraSPARC T1) or eight hardware threads (for the UltraSPARC T2/T2+, and SPARC T3/T4 and later) that act as virtual CPUs. All CPU cores execute instructions concurrently, and each core switches between threads—typically when a thread stalls on a cache miss or goes idle—within a single clock cycle. This lets the processor gain throughput that is lost during cache misses in conventional CPU designs. Each domain is assigned its own CPU threads and executes CPU instructions at native speed, avoiding the virtualization overhead for privileged operation trap-and-emulate or binary rewrite typical of most VM designs.

Each server can support as many as one domain per hardware thread up to a maximum of 128. That's up to 32 domains for the UltraSPARC T1, 64 domains for the UltraSPARC T2 and SPARC T4-1, and 128 domains for UltraSPARC T3 as examples single-processor (single-socket) servers. Servers with 2-4 UltraSPARC T2+ or 2-8 SPARC T3-T5 CPUs support as many logical domains as the number of processors multiplied by the number of threads of each CPU up to the limit of 128.^[11] M-series servers can be subdivided into physical domains ("PDoms"), each of which can host up to 128 logical domains. Typically, a given domain is assigned multiple CPU threads or CPU cores for additional capacity within a single OS instance. CPU threads, RAM, and virtual I/O devices can be added to or removed from a domain by administrator issuing command in the control domain. This change takes effect immediately without the need to reboot the affected domain, which can immediately make use of added CPU threads or continue operating with reduced resources.

When hosts are connected to shared storage (SAN or NAS), running guest domains can be securely live migrated between servers without outage (starting with Oracle VM Server for SPARC version 2.1). The process encrypts guest VM memory contents before they are transmitted between servers, using cryptographic accelerators available on all processors with sun4v architecture.

All logical domains are the same except for the roles that they are assigned. There are multiple roles that logical domains can perform such as:

• Control domain
• Service domain
• I/O domain
• Root domain
• Guest domain

Control domain, as its name implies, controls the logical domain environment. It is used to configure machine resources and guest domains, and provides services necessary for domain operation, such as virtual console service. The control domain also normally acts as a service domain.

Service domains present virtual services, such as virtual disk drives and network switches, to other domains. In most cases, guest domains perform I/O via bridged access through services domains, which are usually I/O domains and directly connected to the physical devices. Service domains can provide virtual LANs and SANs as well as bridge through to physical devices. Disk images can reside on complete local physical disks, shared SAN block devices, their slices, or even on files contained on a local UFS or ZFS file system, or on a shared NFS export or iSCSI target.

Control and service functions can be combined within domains, however it is recommended that user applications not run within control or service domains in order to protect domain stability and performance.

I/O domains have direct ownership of a PCI bus, or card on a bus, or Single Root I/O Virtualization (SR-IOV) function, providing direct access to physical I/O devices, such as a network card in a PCI controller. An I/O domain may use its devices to have native I/O performance its own applications, or act as a service domain and share the devices to other domains as virtual devices.

Root domains have direct ownership of PCIe "root complex" and all associated PCIe slots. This can be used to grant access to physical I/O devices. A root domain is also an I/O domain. There are a maximum of two root domains for the UltraSPARC T1 (Niagara) servers, one of which also must be the control domain. UltraSPARC T2 Plus, SPARC T3, and SPARC T4 servers can have as many as 4 root domains, limited by the number of PCIe root complexes installed on the server. SPARC T5 servers can have up to 16 root complex domains. Multiple I/O domains can be configured to provide resiliency against failures.

Guest domains run an operating system instance without performing any of the above roles, but leverage the services provided by the above in order to run applications.

The only operating system supported by the vendor for running within logical domains is Solaris 10 11/06 and later updates, and all Solaris 11 releases.

There are operating systems that are not officially supported, but may still be capable of running within logical domains:

• Debian ports^[12] version
• OpenSolaris 2009.06
• Illumos-derived releases
• Ubuntu Linux Server Edition
• OpenBSD 4.5 or later^[13]
• Wind River Linux 3.0^[14]
• Oracle Linux for SPARC^[15]

• Oracle VM Server for x86
• Logical partition

• Oracle Announces Latest Version of Oracle VM Server for SPARC
• Oracle VM Server for SPARC product page at Oracle
• Oracle VM Server for SPARC software at Fujitsu
• Increasing Application Availability by Using the Oracle VM Server for SPARC Live Migration Feature
• Logical Domains Community at OpenSolaris.org
• Oracle VM Server for SPARC Best Practices
• Dynamic Resource Management from Oracle.