We study the evolution of the largest known corpus of publicly available source code, i.e., the Software Heritage archive (4B unique source code files, 1B commits capturing their development histories across 50M software projects). On such corpus we quantify the growth rate of original, never-seen-before source code files and commits. We find the growth rates to be exponential over a period of more than 40 years. We then estimate the multiplication factor, i.e., how much the same artifacts (e.g., files or commits) appear in different contexts (e.g., commits or source code distribution places). We observe a combinatorial explosion in the multiplication of identical source code files across different commits. We discuss the implication of these findings for the problem of tracking the provenance of source code artifacts (e.g., where and when a given source code file or commit has been observed in the wild) for the entire body of publicly available source code. To that end we benchmark different data models for capturing software provenance information at this scale and growth rate. We identify a viable solution that is deployable on commodity hardware and appears to be maintainable for the foreseeable future.
With few exceptions @cite_26 , though, file-level provenance has received little attention in the research community. We believe this is due to the lack of a reference archive of public source code on which file-level provenance tracking can be implemented once and then reused by other researchers. In the final part of this paper we discuss the implications of our findings about public source code and explore the feasibility of such provenance service'' approach, relying on as a proxy of public source code.
@cite_26
1858703999 1575826986 2033984447 2162546229
Digital provenance describes the ancestry or history of a digital object. Most existing provenance systems, however, operate at only one level of abstraction: the system call layer, a workflow specification, or the high-level constructs of a particular application. The provenance collectable in each of these layers is different, and all of it can be important. Single-layer systems fail to account for the different levels of abstraction at which users need to reason about their data and processes. These systems cannot integrate data provenance across layers and cannot answer questions that require an integrated view of the provenance. We have designed a provenance collection structure facilitating the integration of provenance across multiple levels of abstraction, including a workflow engine, a web browser, and an initial runtime Python provenance tracking wrapper. We layer these components atop provenance-aware network storage (NFS) that builds upon a Provenance-Aware Storage System (PASS). We discuss the challenges of building systems that integrate provenance across multiple layers of abstraction, present how we augmented systems in each layer to integrate provenance, and present use cases that demonstrate how provenance spanning multiple layers provides functionality not available in existing systems. Our evaluation shows that the overheads imposed by layering provenance systems are reasonable. As increasing amounts of valuable information are produced and persist digitally, the ability to determine the origin of data becomes important. In science, medicine, commerce, and government, data provenance tracking is essential for rights protection, regulatory compliance, management of intelligence and medical data, and authentication of information as it flows through workplace tasks. In this paper, we show how to provide strong integrity and confidentiality assurances for data provenance information. We describe our provenance-aware system prototype that implements provenance tracking of data writes at the application layer, which makes it extremely easy to deploy. We present empirical results that show that, for typical real-life workloads, the run-time overhead of our approach to recording provenance with confidentiality and integrity guarantees ranges from 1 -13 . As organizations become increasingly reliant on cloud computing for servicing their data storage requirements, the need to govern access control at finer granularities becomes particularly important. This challenge is increased by the lack of policy supporting data migration across geographic boundaries and through organizations with divergent regulatory policies. In this paper, we present an architecture for secure and distributed management of provenance, enabling its use in security-critical applications. Provenance, a metadata history detailing the derivation of an object, contains information that allows for expressive, policy-independent access control decisions. We consider how to manage and validate the metadata of a provenance-aware cloud system, and introduce protocols that allow for secure transfer of provenance metadata between end hosts and cloud authorities. Using these protocols, we develop a provenance-based access control mechanism for Cumulus cloud storage, capable of processing thousands of operations per second on a single deployment. Through the introduction of replicated components, we achieve overhead costs of just 14 , demonstrating that provenance-based access control is a practical and scalable solution for the cloud. Existence of data provenance information in a system raises at least two security-related issues. One is how provenance data can be used to enhance security in the system and the other is how to protect provenance data which might be more sensitive than the data itself. Recent data provenance-related access control literature mainly focuses on the latter issue of protecting provenance data. In this paper, we propose a novel provenance-based access control model that addresses the former objective. Using provenance data for access control to the underlying data facilitates additional capabilities beyond those available in traditional access control models. We utilize a notion of dependency as the key foundation for access control policy specification. Dependency-based policy provides simplicity and effectiveness in policy specification and access control administration. We show our model can support dynamic separation of duty, workflow control, origin-based control, and object versioning. The proposed model identifies essential components and concepts and provides a foundational base model for provenance-based access control. We further discuss possible extensions of the proposed base model for enhanced access controls.