language:en region:us
---
language:
- en
pretty_name: sf-nexus-ef-chapters-and-chunks
---

# Dataset Card for SF Nexus Extracted Features: Chapters Only

## Dataset Description

- **Homepage: https://sfnexus.io/** 
- **Repository: https://github.com/SF-Nexus/extracted-features-notebooks** 
- **Point of Contact: Alex Wermer-Colan**

### Dataset Summary

The SF Nexus Extracted Features Chapters and Chunks dataset contains text and metadata from a subset of 306 texts from our corpus of 403 mid-twentieth century science fiction books, originally digitized from Temple University Libraries' Paskow Science Fiction Collection.
After digitization, the books were cleaned using Abbyy FineReader.
Because this is a collection of copyrighted fiction, the books have been disaggregated. 
To improve performance of topic modeling and other nlp tasks, each book has also been split into chapters. This dataset includes the subset of our corpus in which chapters were present.
Each row of this dataset contains one "chapter" of text as well as metadata about that text's title, author and publication.

### About the SF Nexus Corpus
The Paskow Science Fiction collection contains primarily materials from post-WWII, especially mass-market works of the New Wave era (often dated to 1964-1980).
The digitized texts have also been ingested into HathiTrust's repository for preservation and data curation; they are now viewable on HathiTrust's [Temple page](https://babel.hathitrust.org/cgi/ls?field1=ocr;q1=%2A;a=srchls;facet=htsource%3A%22Temple%20University%22;pn=4) for non-consumptive research.
For more information on the project to digitize and curate a corpus of "New Wave" science fiction, see Alex Wermer-Colan's post on the Temple University Scholars Studio blog, ["Building a New Wave Science Fiction Corpus."](https://sites.temple.edu/tudsc/2017/12/20/building-new-wave-science-fiction-corpus/).

### Languages

English

## Dataset Structure

This dataset contains disaggregated "chunks" of text from mid-twentieth century science fiction books and associated metadata. For example: 

```
{'Unnamed': 1,
 'Title': 'THEEARTHISNEAR',
 'Author': 'PESEK',
 'Pub Year': '1973',
 'Chapter': '1',
 'Text': '. . . A But Cadiz. Cape Cape Elijah, Elmo’s Gone Horn, I I Islands, Life, Lisbon, No, Or Palos Prophet So So St The Then Those Verde What Wonderful, a a a a a a a a about above against ago ago! all all along and and and and and and and and and and are aroused as as at at away becalmed beyond beyond beyond black bobbing bows breath breath, breeze broke broke. burning but but but calm came chewed continent continent, corks. crashing dark days days days did down drive dry else, end endless enough even ever far fire fire, flowed foaming for for fresh from from from gave got grew had—a hardly horizon horizon, horizon, hot hungry in in in in in in in it it it it, its itself knew. know lands lay leather, licking life life! life-giving like like little lives. long long longed loose madness masts men middle mist more motionless mouths, nameless nameless nameless names. night. no noon noonday not nothing ocean ocean, ocean, oceans of of of of of of of of of on on on one or our our our our our our our our our out over perhaps price quicksilver remember rigging, roared, roaring, rocks. rose rousing sails sails. salty say? sea sea sea sea shining slack slight smelling some somewhere somewhere spices spices. stayed storm strange strips struck sun. swell swollen tackle tasted teeth terrible, than that that the the the the the the the the the the the the the the the the the the the the the the the the the the their them them, then then then there thin thing, thirst thirst thirst those those though throats. time to to to to tongues tune unknown unknown up, us us wanted was was was was was water water water, water, waves way we we we we weeks, when when when while whistled who who wind wind, without yardarm yards.'
 'Clean Text': ' a but cadiz cape cape elijah elmo s gone horn i i islands life lisbon no or palos prophet so so st the then those verde what wonderful a a a a a a a a about above against ago ago all all along and and and and and and and and and and are aroused as as at at away becalmed beyond beyond beyond black bobbing bows breath breath breeze broke broke burning but but but calm came chewed continent continent corks crashing dark days days days did down drive dry else end endless enough even ever far fire fire flowed foaming for for fresh from from from gave got grew had a hardly horizon horizon horizon hot hungry in in in in in in in it it it it its itself knew know lands lay leather licking life life life giving like like little lives long long longed loose madness masts men middle mist more motionless mouths nameless nameless nameless names night no noon noonday not nothing ocean ocean ocean oceans of of of of of of of of of on on on one or our our our our our our our our our out over perhaps price quicksilver remember rigging roared roaring rocks rose rousing sails sails salty say sea sea sea sea shining slack slight smelling some somewhere somewhere spices spices stayed storm strange strips struck sun swell swollen tackle tasted teeth terrible than that that the the the the the the the the the the the the the the the the the the the the the the the the the the their them them then then then there thin thing thirst thirst thirst those those though throats time to to to to tongues tune unknown unknown up us us wanted was was was was was water water water water waves way we we we we weeks when when when while whistled who who wind wind without yardarm yards'
 'Chapter Word Count': '343',
}
```

### Data Fields

- **Unnamed: int** A unique id for the text
- **Title: str** The title of the book from which the text has been extracted
- **Author: str** The author of the book from which the text has been extracted
- **Pub Year: str** The date on which the book was published (first printing)
- **Chapter: int** The chapter in the book from which the text has been extracted
- **Text: str** The chunk of text extracted from the book
- **Clean Text: str** The chunk of text extracted from the book with lowercasing performed and punctuation, numbers and extra spaces removed
- **Chapter Word Count: int** The number of words the chunk of text contains

To Be Added: 
- **summary: str** A brief summary of the book, if extracted from library records
- **pub_date: int** The date on which the book was published (first printing)
- **pub_city: int** The city in which the book was published (first printing)
- **lcgft_category: str** Information from the Library of Congress Genre/Form Terms for Library and Archival Materials, if known

### Loading the Dataset

Use the following code to load the dataset in a Python environment (note: does not work with repo set to private)
```
from datasets import load_dataset

# If the dataset is gated/private, make sure you have run huggingface-cli login
dataset = load_dataset("SF-Corpus/EF_Chapters_Only")
```
Or just clone the dataset repo

```
git lfs install
git clone https://huggingface.co/datasets/SF-Corpus/EF_Chapters_Only
# if you want to clone without large files – just their pointers
# prepend your git clone with the following env var:
GIT_LFS_SKIP_SMUDGE=1
```

## Dataset Creation

### Curation Rationale

For an overview of our approach to data curation of literary texts, see Alex Wermer-Colan’s and James Kopaczewski’s article, “The New Wave of Digital Collections: Speculating on the Future of Library Curation”(2022)

### Source Data

The Loretta C. Duckworth Scholars Studio has partnered with Temple University Libraries’ Special Collections Research Center (SCRC) and Digital Library Initiatives (DLI) to build a digitized corpus of copyrighted science fiction literature. Besides its voluminous Urban Archives, the SCRC also houses a significant collection of science-fiction literature. The Paskow Science Fiction Collection was originally established in 1972, when Temple acquired 5,000 science fiction paperbacks from a Temple alumnus, the late David C. Paskow. Subsequent donations, including troves of fanzines and the papers of such sci-fi writers as John Varley and Stanley G. Weinbaum, expanded the collection over the last few decades, both in size and in the range of genres. SCRC staff and undergraduate student workers recently performed the usual comparison of gift titles against cataloged books, removing science fiction items that were exact duplicates of existing holdings. A refocusing of the SCRC’s collection development policy for science fiction de-emphasized fantasy and horror titles, so some titles in those genres were removed as well.

## Considerations for Using the Data

This data card only exhibits extracted features for copyrighted fiction; no copyrighted work is being made available for consumption. These digitized files are made accessible for purposes of education and research. Temple University Libraries have given attribution to rights holders when possible. If you hold the rights to materials in our digitized collections that are unattributed, please let us know so that we may maintain accurate information about these materials.

If you are a rights holder and are concerned that you have found material on this website for which you have not granted permission (or is not covered by a copyright exception under US copyright laws), you may request the removal of the material from our site by writing to digitalscholarship@temple.edu.

For more information on non-consumptive research, check out HathiTrust Research Center’s Non-Consumptive Use Research Policy.


## Additional Information

### Dataset Curators
For a full list of conributors to the SF Nexus project, visit [https://sfnexus.io/people/](https://sfnexus.io/people/). 