CP Tutorial
________ Overview ____________________________________________________
A pickle is a serialized and closed representation of a value, stored in a file, for instance. Pickles can be used to exchange arbitrary data structures between processes. Pickles may contain higher-order values (i.e., functions) as well as complete modules (e.g., implementations of abstract types). Pickling is type-safe: a pickle is in fact a serialized representation of a package, a pair of a value and its type. Types are checked at unpickling time.
________ Semantics ___________________________________________________
When a value is pickled, this comprehends the transitive closure of all objects referenced by that value. We distinguish three sorts of values:
Functional values do not contain any mutable objects. Functional values can be pickled. Unpickling a functional value results in an object indistinguishable from the original object.
Stateful values do contain mutable objects like references or arrays. Stateful values can also be pickled. In contrast to functional values, unpickling a stateful value always returns a fresh copy of state the object was in at pickling time. Sharing between stateful objects within a single pickle is maintained.
Sited values contain resources. A resource is an object that cannot be meaningfully interpreted outside of the process it was created in, or that represents a security-relevant capability. Sited values may not be pickled: any attempt to pickle a sited value results in a runtime exception. This is necessary to enable security and sandboxing.
Special care must be taken to avoid runtime errors from attempts to pickle sited values. Resources may be operating system handles (for example, open files) or computational resources (for example, first-class threads). Higher-order values may be sited without this being visible in their type, for instance, when a function closure references a sited object. Moreover, all functions that create resources are themselves sited.
Futures are never pickled. Instead, pickling requests all futures in the value to be pickled.
________ Persistent modules __________________________________________
Pickling is available through the structure Pickle. The canonical operation to create a pickle is the operation
save : string * package -> unit
For example, the Int module can be exported as a pickle as follows:
Pickle.save("Int." ^ Pickle.extension, pack Int : INTEGER)
The package will be serialized to a file with the specified name. The string Pickle.extension gives the file extension idiomatically used for pickles on the current platform. If the module contained references to any sited objects, an IO.Io exception would be raised, with Sited indicating the cause of the failure (Int is not sited, however).
The inverse operation is unpickling, available through the operation
load : string -> package
structure Int' = unpack Pickle.load("Int." ^ Pickle.extension) : INTEGER
If unpickling is successful, Int' will be accessible as a structure with signature INTEGER. Loading of a pickle may fail with an IO.Io exception.
Pickled modules can contain abstract types. Sometimes it is necessary to express sharing between abstract types of different pickles. The way to deal with this is using appropriate type constraints upon unpacking the loaded package. For example, consider an abstract datatype
signature T =
val mk : int -> t
val f : t -> int
that is stored in a pickle p1. Another pickle p2 contains a value of that type. Both can be loaded and used together:
structure T = unpack p1 : T
structure V = unpack p2 : (val x : T.t)
val n = T.f V.x
See also the section on sharing across packages.
________ Pickles and components ______________________________________
Pickles are closely related to components. In fact, a pickle is the special case of an evaluated component, residing in a file. Pickles can thus be imported through import announcements.
The load operations for pickles can access arbitrary components. The file name argument is interpreted as an URI. If the file at the designated location does not contain a plain pickle, the component will be linked and evaluated by the current component manager and its export is returned. This may cause arbitrary side effects. In particular, it may trigger the loading of additional, transitively imported components, if their exports are required.
Note that evaluation of components causes the creation of new generative types. Thus, if a non-pickle component is loaded twice, any generative types contained in its export will be incompatible between both instances. The same holds for exceptions.