OpenVMS

OpenVMS, often referred to as just VMS, is a multi-user, multiprocessing virtual memory-based operating system designed to support time-sharing, batch processing, transaction processing and workstation applications. It was first announced by Digital Equipment Corporation as VAX/VMS ("Virtual Address eXtension/Virtual Memory System") alongside the VAX-11/780 minicomputer in 1977. OpenVMS has subsequently been ported to run on DEC Alpha systems, the Itanium-based HPE Integrity family of computers, and select x86-64 hardware and hypervisors. Since 2014, OpenVMS is developed and supported by a company named VMS Software Inc. (VSI).
The system offers high availability through clustering and the ability to distribute the system over multiple physical machines, allowing clustered applications and data to remain continuously accessible while operating system software and hardware maintenance and upgrades are performed, or when a whole data center is destroyed. VMS cluster uptimes of up to 17 years have been reported. Customers using OpenVMS include banks and financial services, hospitals and healthcare, telecommunications operators, network information services, and industrial manufacturers. During the 1990s and 2000s, there were approximately half a million VMS systems in operation worldwide.
History.
Origin and name changes.
In April 1975, Digital Equipment Corporation embarked on a hardware project, code named "Star", to design a 32-bit virtual address extension to its PDP-11 computer line. A companion software project, code named "Starlet", was started in June 1975 to develop a totally new operating system, based on RSX-11M, for the Star family of processors. These two projects were tightly integrated from the beginning. Gordon Bell was the VP lead on the VAX hardware and its architecture. Roger Gourd was the project lead for the Starlet program, with software engineers Dave Cutler (who would later lead development of Microsoft's Windows NT), Dick Hustvedt, and Peter Lipman acting as the technical project leaders, each having responsibility for a different area of the operating system. The Star and Starlet projects culminated in the VAX-11/780 computer and the VAX/VMS operating system. The Starlet name survived in VMS as a name of several of the main system libraries, including codice_1 and codice_2.
With the introduction of the MicroVAX range such as the MicroVAX I, MicroVAX II and MicroVAX 2000 in the mid-to-late 1980s, DIGITAL released MicroVMS versions specifically targeted for these platforms which had much more limited memory and disk capacity. MicroVMS kits were released for VAX/VMS 4.0 to 4.7 on TK50 tapes and RX50 floppy disks, but discontinued with VAX/VMS V5.0.
Beginning in 1989, a short lived distribution of VMS named "Desktop-VMS" was sold with VAXstation systems. It consisted of a bundle of VMS, DECwindows, DECnet, VAXcluster support, and software that was designed to help non-technical users set up their workstation without assistance. Desktop-VMS had its own versioning scheme beginning with V1.0, which corresponded to the V5.x releases of VAX/VMS.
In July 1992, Digital renamed VAX/VMS to OpenVMS as an indication for its support of "open systems" industry standards such as POSIX and Unix compatibility, and to drop the VAX connection since the port to Alpha was underway. The OpenVMS name was first used with the OpenVMS AXP V1.0 release in November 1992. Digital began using OpenVMS VAX instead of VAX/VMS with the V6.0 release in June 1993.
Port to DEC Alpha.
During the 1980s, Digital planned to replace the VAX platform and the VMS operating system with the PRISM architecture and the MICA operating system. When these projects were cancelled in 1988, a team was set up to design new VAX/VMS systems of comparable performance to RISC-based Unix systems. After a number of failed attempts to design a faster VAX-compatible processor, the group demonstrated the feasibility of porting VMS and its applications to a RISC architecture based on PRISM. This led to the creation of the Alpha architecture. The project to port VMS to Alpha began in 1989, and first booted on a prototype Alpha EV3-based "Alpha Demonstration Unit" in early 1991. Prior to the availability of Alpha hardware, OpenVMS was developed and booted on an emulator named "Mannequin", which implemented many of the Alpha instructions in custom microcode on a VAX 8800 system. 
The main challenge in porting VMS to a new architecture was that VMS and the VAX were designed together, meaning that VMS was dependent on certain details of the VAX architecture. Furthermore, a significant amount of the VMS kernel, layered products, and customer-developed applications were implemented in VAX MACRO assembly code. Some of the changes needed to decouple VMS from the VAX architecture included:
The VMS port to Alpha resulted in the creation of two separate source code libraries (based on a source code management tool known as the VMS Development Environment, or VDE) for VAX, and for Alpha. The Alpha code library was based on a snapshot of the VAX/VMS code base circa V5.4-2. 1992 saw the release of the first version of OpenVMS for Alpha AXP systems, designated "OpenVMS AXP V1.0". In 1994, with the release of OpenVMS V6.1, feature (and version number) parity between the VAX and Alpha variants was achieved, this was the so-called Functional Equivalence release. The decision to use the 1.x version numbering stream for the pre-production quality releases of OpenVMS AXP caused confusion for some customers, and was not repeated in the subsequent ports of OpenVMS to new platforms.
When VMS was ported to Alpha, it was initially left as a 32-bit only operating system. This was done to ensure backwards compatibility with software written for the 32-bit VAX. 64-bit addressing was first added for Alpha in the V7.0 release. In order to allow 64-bit code to interoperate with older 32-bit code, OpenVMS does not create a distinction between 32-bit and 64-bit executables, but instead allows for both 32-bit and 64-bit pointers to be used within the same code. This is known as mixed pointer support. The 64-bit OpenVMS Alpha releases support a maximum virtual address space size of 8TiB (a 43-bit address space), which is the maximum supported by the Alpha 21064 and Alpha 21164.
One of the more noteworthy Alpha-only features of OpenVMS was "OpenVMS Galaxy" - which allowed the partitioning of a single SMP server to run multiple instances of OpenVMS. Galaxy supported dynamic resource allocation to running partitions, and the ability to share memory between partitions.
Port to Intel Itanium.
In 2001, just prior to its acquisition by Hewlett-Packard, Compaq announced the port of OpenVMS to the Intel Itanium architecture. The Itanium port was the result of Compaq's decision to discontinue future development of the Alpha architecture in favour of adopting the then-new Itanium architecture. The porting began in late 2001, and the first boot on took place on the 31st of January 2003. The first boot consisted of booting a minimal system configuration on a HP i2000 workstation, logging in as the codice_3 user, and running the codice_4 command.
The Itanium port was accomplished using source code maintained in common within the OpenVMS Alpha source code library, with the addition of conditional code and additional modules where changes specific to Itanium were required. Whereas the VAX and Alpha architectures were specifically designed to support the low-level needs of OpenVMS, Itanium was not. This required certain architectural dependencies of OpenVMS to be replaced, or emulated in software. Some of the changes included:
As with the VAX to Alpha port, a binary translator for Alpha to Itanium was made available, allowing user mode OpenVMS Alpha software to be ported to Itanium in situations where it was not possible to recompile the source code. This translator is known as the Alpha Environment Software Translator (AEST), and it also supported translating VAX executables which had already translated with VEST.
The Itanium port was officially named "HP OpenVMS Industry Standard 64 for Integrity Servers", although the name "OpenVMS I64" is more commonly used. Two pre-production releases, OpenVMS I64 V8.0 and V8.1, were available on June 30, 2003 and on December 18, 2003. These releases were intended for HP organizations and third-party vendors involved with porting software packages to OpenVMS I64. The first production release, V8.2, was released in February 2005. V8.2 was also released for Alpha, subsequent V8.x releases of OpenVMS have maintained feature parity between the Alpha and Itanium architectures.
Port to x86-64.
When VMS Software Inc. (VSI) announced that they had secured the rights to develop the OpenVMS operating system from HP, they also announced their intention to port OpenVMS to the x86-64 architecture. The porting effort ran concurrently with the establishment of the company, as well as the development of VSI's own Itanium and Alpha releases of OpenVMS 8.x.
The x86-64 port is targeted for specific servers from HPE and Dell, as well as certain virtual machine hypervisors. Initial support was targeted for KVM and VirtualBox. Support for VMware was announced in 2020, and Hyper-V has been described as a future target.
The x86-64 port is built from the same source code library as the Alpha and Itanium architectures, using conditional compilation to manage the architecture-specific code needed to support the x86-64 platform. As with the Alpha and Itanium ports, the x86-64 port made some changes to simplify porting and supporting OpenVMS on the new platform:
The first boot was announced on 14 May 2019. This involved booting OpenVMS on VirtualBox, and successfully running the codice_4 command. Later in 2019, the first "real boot" was announced - this consisted of the operating system booting in a completely standard manner, a user logging into the system, and running the codice_4 command. In May 2020, the V9.0 Early Adopter's Kit release was made available to a small number of customers. This consisted of the OpenVMS operating system running in a VirtualBox VM with certain limitations - most significantly, few layered products were available, and code can only be compiled for x86-64 using cross compilers which run on Itanium-based OpenVMS systems. Following the V9.0 release, VSI released a series of updates on a monthly or bimonthly basis which added additional functionality and hypervisor support. These were designated V9.0-A through V9.0-H.
Architecture.
The OpenVMS operating system has a layered architecture, consisting of a privileged "Executive", a "Command Language Interpreter" which runs at an intermediate level of privilege, and utilities and run-time libraries (RTLs) which run in an unprivileged mode, but can potentially run at a higher level of privilege if authorized to do so. Unprivileged code typically invokes the functionality of the Executive through "system services" (equivalent to system calls in other operating systems).
OpenVMS' layers and mechanisms are built around certain features of the VAX architecture, including:
These VAX architecture mechanisms are implemented on Alpha, Itanium and x86-64 by either mapping to corresponding hardware mechanisms on those architectures, or through emulation (via PALcode on Alpha, or in software on Itanium and x86-64).
Executive and Kernel.
The OpenVMS Executive comprises the privileged code and data structures which reside in the system space. The Executive is further subdivided between the "Kernel", which consists of the code which runs at the kernel access mode, and the less-privileged code outside of the Kernel which runs at the executive access mode.
The components of the Executive which run at executive access mode include the Record Management Services, and certain system services such as image activation. The main distinction between the kernel and executive access modes is that most of the operating system's core data structures can be read from executive mode, but require kernel mode to be written to. Code running at executive mode can switch to kernel mode at will, meaning that the barrier between the kernel and executive modes is intended as a safeguard against accidental corruption as opposed to a security mechanism.
The Kernel compromises the operating system's core data structures (e.g. page tables, the I/O database and scheduling data), and the routines which operate on these structures. The Kernel is typically described as having three major subsystems: I/O, Process and Time Management, Memory Management. In addition, other functionality such as logical name management, synchronization and system service dispatch are implemented inside the Kernel.
Executive structure.
In early versions of VAX/VMS, most of the Executive's code was linked into a single executable image named codice_7. VAX/VMS 5.0 introduced the "Modular Executive", which split the Executive code into a number of "executive images" (also known as "execlets") which are loaded during system bootstrap. codice_7 remained, but was reduced to system service dispatch vectors, static memory locations for data common to multiple executive images, and some basic support code. On OpenVMS for Alpha, Itanium and x86-64, codice_7 is further subdivided into codice_10 and codice_11, which contain the shared memory locations and support code, and the system service dispatch logic, respectively.
Extension mechanisms.
OpenVMS allows user mode code with suitable privileges to switch to executive or kernel mode using the codice_12 and codice_13 system services, respectively. This allows code outside of system space to have direct access to the Executive's routines and system services. In addition to allowing third-party extensions to the operating system, Privileged Images are used by core operating system utilities to manipulate operating system data structures through undocumented interfaces.
OpenVMS also allows Shareable Images (i.e. shared libraries) to be granted privilege, allowing the creation of "user-written system services", which are privileged routines which can be linked into a non-privileged program. User written system services are invoked using the same mechanism as standard system services, which prevents the unprivileged program from gaining the privileges of the code in the Privileged Shareable Image. Despite what the name may suggest, user-written system services are also used to implement seldom-used operating system functionality such as volume mounting.
File system.
OpenVMS provides feature-rich facilities for file management. The typical user and application interface into the file system is via the Record Management Services (RMS), although applications can interface directly with the underlying file system through the QIO system services. RMS supports multiple record-oriented file access methods and record formats (including a stream format where the file is treated as a stream of bytes, similar to Unix). RMS also supports remote file access via DECnet, and optional support for journaling.
The file systems supported by VMS are referred to as the Files-11 On-Disk Structures (ODS), which provide disk quotas, access control lists and file versioning. The most significant structure levels are ODS-2, which is the original VMS file system, and ODS-5, which extended ODS-2 with support for Unicode file names, case sensitivity, hard links and symbolic links. VMS is also capable of accessing files on ISO 9660 CD-ROMs and magnetic tape with ANSI tape labels.
Alongside the OpenVMS Alpha V7.0 release in 1995, Digital released a log-structured file system named Spiralog which was intended as a potential successor to Files-11. Spiralog shipped as an optional product, and was discontinued at the release of OpenVMS Alpha 7.2. Spiralog's discontinuation was due to a variety of problems, including issues with handling full volumes. The developers of Spiralog began work on a new file system in 1996, which was put on hold and later resumed by VSI in 2016 as the VMS Advanced File System (VAFS, not to be confused with Digital's AdvFS for Tru64). VAFS no longer appears on recent roadmaps, and instead VSI have discussed porting the open source GFS2 file system to OpenVMS. One of the major motivations for replacing the Files-11 structures is that they are limited to 2TiB volumes.
Command Language Interpreter.
An OpenVMS Command Language Interpreter (CLI) implements a command line interface for OpenVMS; responsible for executing individual commands, as well as command procedures (equivalent to shell scripts or batch files). The standard CLI for OpenVMS is the DIGITAL Command Language, although other options are available as well.
Unlike Unix shells, which typically run in their own isolated process and behave like any other user mode program, OpenVMS CLIs are an optional component of a process, which exist alongside any executable image which that process may run. Whereas a Unix shell will typically run executables by creating a separate process using fork-exec, an OpenVMS CLI will typically load the executable image into the same process, transfer control to the image, and ensure that control is transferred back to CLI once the image has exited and that the process is returned to its original state. A CLI gets mapped into a process' private address space through execution of the codice_14 image, which can either be executed manually, or automatically by certain process creation system services.
Due to fact that the CLI is loaded into the same address space as user code, and that the CLI is responsible for invoking image activation and image rundown, the CLI is mapped into memory at supervisor access mode. This is in order to prevent accidental or malicious manipulation of the CLI's code and data structures by user mode code.
Features.
User interfaces.
VMS was originally designed to be used and managed interactively using Digital's text-based video terminals such as the VT100, or hardcopy terminals such as the DECwriter series. Since the introduction of the VAXstation line in 1984, VMS has optionally supported graphical user interfaces for use with workstations or X terminals.
Command line interfaces.
The DIGITAL Command Language has served as the primary Command Language Interpreter (CLI) of OpenVMS since the first release. Other official CLIs available for VMS include the RSX-11 MCR (VAX only), and various Unix shells. Digital provided tools for creating text-based user interface applications – the Form Management System (FMS) and Terminal Data Management System (TDMS), later succeeded by DECforms. A lower level library, comparable to Unix curses called the Screen Management Services (SMG$) also exists.
Graphical user interfaces.
Over the years, VMS has gone through a number of different GUI toolkits and interfaces:
Versions of VMS running on DEC Alpha workstations in the 1990s supported OpenGL and Accelerated Graphics Port (AGP) graphics adapters. VMS also provides support for older graphics standards such as GKS and PHIGS. Modern versions of DECwindows are based on X.org Server.
Clustering.
OpenVMS supports clustering (first called "VAXcluster" and later "VMScluster"), where multiple systems run their own instance of the operating system, but share disk storage, processing, a distributed lock manager, a common management and security domain, job queues and print queues, providing a single system image abstraction. The systems are connected either by proprietary specialized hardware (Cluster Interconnect) or an industry-standard Ethernet LAN. OpenVMS supports up to 96 nodes in a single cluster, and allows mixed-architecture clusters, where VAX and Alpha systems, or Alpha and Itanium systems can co-exist in a single cluster. VMS clusters allow the creation of applications which can withstand planned or unplanned outages of part of the cluster.
Networking.
Digital's DECnet protocol suite is tightly integrated into VMS, allowing remote logins, as well as transparent access to files, printers and other resources on VMS systems over a network. Modern versions of VMS support both the traditional Phase IV DECnet protocol, as well the OSI-compatible Phase V (also known as DECnet-Plus). Support for TCP/IP is provided by the optional TCP/IP Services for OpenVMS layered product (originally known as the VMS/ULTRIX Connection, then as the ULTRIX Communications Extensions or UCX). TCP/IP Services is based on a port of the BSD network stack to OpenVMS, along with support for common protocols such as SSH, DHCP, FTP and SMTP. Due to the fact that the official TCP/IP stack was introduced relatively late, multiple third party TCP/IP stacks have been created for VMS. Some of these third party TCP/IP stacks remain under active development, such as TCPware and MultiNet.
Digital sold a software package named PATHWORKS (originally known as the Personal Computer Systems Architecture or PCSA) which allowed personal computers running MS-DOS, Microsoft Windows or OS/2, or the Apple Macintosh to serve as a terminal for VMS systems, or to use VMS systems as a file or print server. PATHWORKS was based on LAN Manager and supported either DECnet or TCP/IP as a transport protocol. PATHWORKS was later renamed to Advanced Server for OpenVMS, and was eventually replaced with a VMS port of Samba at the time of the Itanium port.
Digital provided the Local Area Transport (LAT) protocol which allowed remote terminals and printers to be attached to a VMS system through a terminal server.
Programming.
Digital (and its successor companies) provided a wide variety of programming languages for VMS. Officially supported languages on VMS, either current or historical, include:
Among OpenVMS's notable features is the Common Language Environment, a strictly defined standard that specifies calling conventions for functions and routines, including use of stacks, registers, etc., independent of programming language. Because of this, it is possible to call a routine written in one language (for example, Fortran) from another (for example, COBOL), without needing to know the implementation details of the target language. OpenVMS itself is implemented in a variety of different languages and the common language environment and calling standard supports freely mixing these languages. Digital created a tool named the Structure Definition Language (SDL), which allowed data type definitions to be generated for different languages from a common definition.
Development Tools.
Digital provided a collection of software development tools in a layered product named DECset (originally named VAXset). This consisted of the Language-Sensitive Editor (LSE), a version control system (the Code Management System or CMS), a build tool (the Module Management System or MMS), a static analyzer (the Source Code Analyzer or SCA), a profiler (the Performance and Coverage Analyzer or PCA) as well as a test manager (the Digital Test Manager or DTM). In addition, a number of text editors are included in the operating system, including EDT, EVE and TECO.
The OpenVMS Debugger supports all DEC compilers and many third party languages. It allows breakpoints, watchpoints and interactive runtime program debugging either using a command line or graphical user interface. A pair of lower-level debuggers, called DELTA and XDELTA, can be used to debug privileged code in additional to normal application code.
In 2019, VSI released an officially-supported Integrated Development Environment for VMS based on Visual Studio Code. This allows VMS applications to be developed and debugged remotely from a Microsoft Windows, macOS or Linux workstation.
Database management.
Digital created a number of optional database products for VMS, some of which were marketed as the "VAX Information Architecture" family. These products included:
In 1994, Digital sold Rdb, DBMS and CDD to Oracle, where they remain under active development. In 1995, Digital sold DSM to InterSystems, who renamed it Open M, and eventually replaced it with their Caché product.
Examples of third-party database management systems for OpenVMS include MariaDB, Mimer SQL and System 1032.
Security.
OpenVMS provides various security features and mechanisms, including security identifiers, resource identifiers, subsystem identifiers, ACLs, intrusion detection and detailed security auditing and alarms. Specific versions evaluated at DoD NCSC Class C2 and, with the SEVMS security enhanced services support, at NCSC Class B1, per the NCSC Rainbow Series. OpenVMS also holds an ITSEC E3 rating (see NCSC and Common Criteria). Passwords are hashed using the Purdy Polynomial.
Vulnerabilities.
A 33-year-old vulnerability in VAX/VMS and OpenVMS Alpha, was discovered in 2017 and assigned the CVE ID . On the affected platforms, this vulnerability allowed an attacker with access to the DCL command line to bypass system security and take full control of the system. The vulnerability relies on exploiting a buffer overflow bug in the DCL command processing code, the ability for a user to interrupt a running image (program executable) with CTRL/Y and return to the DCL prompt, and the fact that DCL retains access to the privileges of the images that it requests to be loaded into the DCL process when the image is interrupted. The buffer overflow bug allowed shellcode to be executed with the privileges of an interrupted image, and thus allowing an attacker to carry out a privilege escalation attack.
Cross platform compatibility.
VAX/VMS originally included an RSX-11M compatibility layer named the RSX Application Migration Executive (AME) which allowed user mode RSX-11M software to be run unmodified on top of VMS. This relied on the PDP-11 compatibility mode implemented in the VAX-11 processors. The RSX AME played an important role on early versions of VAX/VMS, which used re-used certain RSX-11M user space utilities before native VAX versions had been developed. This was discontinued in VAX/VMS V3.0 when all compatibility mode utilities were replaced with native implementations, and RSX AME was removed from the base system. At this point, RSX AME was replaced with an optional layered product on VAX named VAX-11 RSX, which relied on software emulation to run PDP-11 code on newer VAX processors. A VAX port of the RTEM compatibility layer for RT-11 applications was also available from Digital.
Various official Unix compatibility layers were created for VMS. The first of which was DEC/Shell - which was a layered product consisting of port of the Version 7 Unix Bourne Shell and several other Unix utilities to VAX/VMS. In 1992, Digital released the POSIX for OpenVMS layered product, which included a shell based on the Korn Shell. POSIX for OpenVMS was later replaced by the open source GNV (GNU's not VMS) project, which was first included in OpenVMS media in 2002. Amongst other GNU tools, GNV includes a port of the Bash shell to VMS. Examples of third party Unix compatibility layers for VMS include Eunice.
Digital licensed SoftPC (and later SoftWindows), and sold it as a layered product for both the VAX and Alpha architectures, allowing Windows and DOS applications to run on top of VMS.
In 1995, Digital announced Affinity for OpenVMS (also known as NT Affinity) which was intended to allow OpenVMS to serve as the persistence layer for Windows NT client-server applications. As part of this initiative, an implementation of the Distributed Component Object Model (DCOM) was added to OpenVMS Alpha, which first appeared in the V7.2-1 release.
Open source applications.
Some of the open source applications which have been ported to OpenVMS include:
There are a number of community projects to port open source software to VMS, including VMS-Ports and GNV (GNU's Not VMS).
Hobbyist programs.
Despite being a proprietary commercial operating system, in 1997 OpenVMS and a number of layered products were made available free of charge for hobbyist, non-commercial use as part of the OpenVMS Hobbyist Program. Since then, several companies producing OpenVMS software have made their products available under the same terms, such as Process Software and MVP Systems. In 2012, HP staff took over the administration of the hobbyist licences. Registration was simplified, and software kits for operating system and layered products were made available on request via FTP download (previously it had to be shipped on CD).
In March 2020, HPE announced that they were concluding the OpenVMS Hobbyist license program. This was followed by an announcement from VSI in April 2020 that VSI they would launch a Community License Program (CLP) to replace the old Hobbyist Program. The CLP was launched in July 2020, and provides licenses for VSI OpenVMS releases on Alpha and Integrity systems. OpenVMS x86-64 licenses will be available later as a more stable version is released for this architecture. OpenVMS for VAX is not covered by the CLP, since there are no VSI releases of OpenVMS VAX, and the old versions are still owned by HPE.
Influence.
During the 1980s, the MICA operating system for the PRISM architecture was intended to be the eventual successor to VMS. MICA was designed to maintain a strong degree of backwards compatibility with VMS applications while also supporting Ultrix applications on top of the same kernel. MICA was ultimately cancelled along with the rest of the PRISM platform, leading Dave Cutler to leave Digital for Microsoft. At Microsoft, Cutler led the creation of the Windows NT operating system, which was heavily inspired by the architecture of MICA. As a result, VMS is considered an ancestor of Windows NT, together with RSX-11, VAXELN and MICA, and many similarities exist between VMS and NT. This lineage is made clear in Cutler's foreword to "Inside Windows NT" by Helen Custer.
FreeVMS was an attempt to develop an open source operating system following VMS conventions. the associated mailing list had been totally inactive for two years and shown limited activity for some years prior to that. FreeVMS supported the x86-64 architecture using an L4 microkernel.