9954967
Petabyte Scale Data Mining: Dream or Reality?
Science is becoming very data intensive. Today's astronomy datasets with tens of millions of galaxies already present substantial challenges for data mining. In less than 10 years the catalogs are expected to grow to billions of objects, and image archives will reach Petabytes. Imagine having a 100GB database in 1996, when disk scanning speeds were 30MB/s, and database tools were immature. Such a task today is trivial, almost manageable with a laptop. We think that the issue of a PB database will be very similar in six years. In this paper we scale our current experiments in data archiving and analysis on the Sloan Digital Sky Survey data six years into the future. We analyze these projections and look at the requirements of performing data mining on such data sets. We conclude that the task scales rather well: we could do the job today, although it would be expensive. There do not seem to be any show-stoppers that would prevent us from storing and using a Petabyte dataset six years from today.
17368591
The Sloan Digital Sky Survey Science Archive: Migrating a Multi-Terabyte Astronomical Archive from Object to Relational DBMS
The Sloan Digital Sky Survey Science Archive is the first in a series of multi-Terabyte digital archives in Astronomy and other data-intensive sciences. To facilitate data mining in the SDSS archive, we adapted a commercial database engine and built specialized tools on top of it. Originally we chose an object-oriented database management system due to its data organization capabilities, platform independence, query performance and conceptual fit to the data. However, after using the object database for the first couple of years of the project, it soon began to fall short in terms of its query support and data mining performance. This was as much due to the inability of the database vendor to respond our demands for features and bug fixes as it was due to their failure to keep up with the rapid improvements in hardware performance, particularly faster RAID disk systems. In the end, we were forced to abandon the object database and migrate our data to a relational database. We describe below the technical issues that we faced with the object database and how and why we migrated to relational technology.
19318578
Extending the SDSS Batch Query System to the National Virtual Observatory Grid
The Sloan Digital Sky Survey science database is approaching 2TB. While the vast majority of queries normally execute in seconds or minutes, this interactive execution time can be disproportionately increased by a small fraction of queries that take hours or days to run; either because they require non-index scans of the largest tables or because they request very large result sets. In response to this, we added a multi-queue job submission and tracking system. The transfer of very large result sets from queries over the network is another serious problem. Statistics suggested that much of this data transfer is unnecessary; users would prefer to store results locally in order to allow further cross matching and filtering. To allow local analysis, we implemented a system that gives users their own personal database (MyDB) at the portal site. Users may transfer data to their MyDB, and then perform further analysis before extracting it to their own machine. 
We intend to extend the MyDB and asynchronous query ideas to multiple NVO nodes. This implies development, in a distributed manner, of several features, which have been demonstrated for a single node in the SDSS Batch Query System (CasJobs). The generalization of asynchronous queries necessitates some form of MyDB storage as well as workflow tracking services on each node and coordination strategies among nodes.
14193738
Cosmological Parameters from Eigenmode Analysis of Sloan Digital Sky Survey Galaxy Redshifts
We present estimates of cosmological parameters from the application of the Karhunen-Loeve transform to the analysis of the three-dimensional power spectrum of density fluctuations using Sloan Digital Sky Survey galaxy redshifts. We use Ωmh and fb = Ωb/Ωm to describe the shape of the power spectrum, σ for the (linearly extrapolated) normalization, and β to parameterize linear theory redshift-space distortions. On scales k 0.16 h Mpc-1, our maximum likelihood values are Ωmh = 0.264 ± 0.043, fb = 0.286 ± 0.065, σ = 0.966 ± 0.048, and β = 0.45 ± 0.12. When we take a prior on Ωb from the Wilkinson Microwave Anisotropy Probe (WMAP), we find Ωmh = 0.207 ± 0.030, which is in excellent agreement with WMAP and the Two-Degree Field. This indicates that we have reasonably measured the gross shape of the power spectrum, but we have difficulty breaking the degeneracy between Ωmh and fb, because the baryon oscillations are not resolved in the current spectroscopic survey window function.
12139365
TeraScale SneakerNet: Using Inexpensive Disks for Backup, Archiving, and Data Exchange
Large datasets are most economically trnsmitted via parcel post given the current economics of wide-area networking. This article describes how the Sloan Digital Sky Survey ships terabyte scale datasets both within the US and to Europe and Asia. We 3GT storage bricks (Ghz processor, GB ram, GbpsEthernet, TB disk) for about 2k$ each. These bricks act as database servers on the LAN. They are loaded at one site and read at the second site. The paper describes the bricks, their economics, and some software issues that they raise.
3204824
Designing a multi-petabyte database for LSST
The 3.2 giga-pixel LSST camera will produce approximately half a petabyte of archive images every month. These data need to be reduced in under a minute to produce real-time transient alerts, and then added to the cumulative catalog for further analysis. The catalog is expected to grow about three hundred terabytes per year. The data volume, the real-time transient alerting requirements of the LSST, and its spatio-temporal aspects require innovative techniques to build an efficient data access system at reasonable cost. As currently envisioned, the system will rely on a database for catalogs and metadata. Several database systems are being evaluated to understand how they perform at these data rates, data volumes, and access patterns. This paper describes the LSST requirements, the challenges they impose, the data access philosophy, results to date from evaluating available database technologies against LSST requirements, and the proposed database architecture to meet the data challenges.
119055256
An explicit example of a moduli driven phase transition in heterotic models
We present an explicit example of a gauge symmetry breaking phase transition in heterotic models, the dynamics of which are not thermal and can be described in a well controlled manner throughout. The phase transition is driven by the evolution of bundle moduli--moduli associated with gauge field vacuum expectation values in the hidden dimensions. We present the necessary parts of the four dimensional effective theory including moduli which describe the embedding of the gauge bundle within the gauge group. We then present exact cosmological solutions to the system before going on to use them to describe the phase transition. The explicit nature of our description enables us to plot how the gauge bosons associated with the symmetries which are broken in the transition gain masses with time. This is in contrast to the use, for example, of a brane collision as a modulus driven phase transition. In the course of this work we find a number of other interesting results. We observe that the Kaehler potential of the system is given by the logarithm of the volume of the compact space even when bundle moduli are included. We also note that the dynamics of the gauge bundle mean that smallmore » instanton transitions are classically forbidden for all but a set of measure zero of the initial conditions of the system. The paper is written in such a manner that the cosmological description of the phase transition can be read independently of the derivation of the four dimensional theory.« less
11407796
Flux, gaugino condensation, and antibranes in heterotic M theory
We present the potential energy due to flux and gaugino condensation in heterotic M theory compactifications with antibranes in the vacuum. For reasons which we explain in detail, the contributions to the potential due to flux are not modified from those in supersymmetric contexts. The discussion of gaugino condensation is, however, changed by the presence of antibranes. We show how a careful microscopic analysis of the system allows us to use standard results in supersymmetric gauge theory in describing such effects - despite the explicit supersymmetry breaking which is present. Not surprisingly, the significant effect of antibranes on the threshold corrections to the gauge kinetic functions greatly alters the potential energy terms arising from gaugino condensation.
119425527
Moving five-branes in low-energy heterotic M theory
We construct cosmological solutions of four-dimensional effective heterotic M-theory with a moving five-brane and evolving dilaton and T modulus. It is shown that the five-brane generates a transition between two asymptotic rolling-radii solutions. Moreover, the five-brane motion always drives the solutions towards strong coupling asymptotically. We present an explicit example of a negative-time branch solution which ends in a brane collision accompanied by a small-instanton transition. The five-dimensional origin of some of our solutions is also discussed.
18073631
A Comprehensive Scan for Heterotic SU(5) GUT models
A bstractCompactifications of heterotic theories on smooth Calabi-Yau manifolds remain one of the most promising approaches to string phenomenology. In two previous papers, arXiv:1106.4804 and arXiv:1202.1757, large classes of such vacua were constructed, using sums of line bundles over complete intersection Calabi-Yau manifolds in products of projective spaces that admit smooth quotients by finite groups. A total of 1012 different vector bundles were investigated which led to 202 SU(5) Grand Unified Theory (GUT) models. With the addition of Wilson lines, these in turn led, by a conservative counting, to 2122 heterotic standard models. In the present paper, we extend the scope of this programme and perform an exhaustive scan over the same class of models. A total of 1040 vector bundles are analysed leading to 35, 000 SU(5) GUT models. All of these compactifications have the right field content to induce low-energy models with the matter spectrum of the supersymmetric standard model, with no exotics of any kind. The detailed analysis of the resulting vast number of heterotic standard models is a substantial and ongoing task in computational algebraic geometry.
55130881
The Replacement of the ISIS White-Circuit Choke
The ISIS accelerator is based on an 800 MeV synchrotron arranged as a White-Circuit [1]. Key to its operation is a large storage choke with ten main, and ten auxiliary windings. The main windings resonate with the associated capacitor banks at 50 Hz and also provide a path for a DC bias current past the capacitors. The auxiliary windings provide for the AC excitation of the magnet circuit. Known as the “NINA” choke from its previous service on the NINA synchrotron at Daresbury Laboratory, it was built in 1963. The windings are built into a single frame and are therefore closely coupled by the magnetic circuit. In the event of a failure on any winding, the choke would have to be dismantled to replace the damaged parts. As it weighs about 90 tons without its cooling oil this would be a major task even if sufficient spare parts were available.