task_categories:text-classification task_categories:summarization task_categories:question-answering language:en license:unknown text classification clustering newsgroups region:us
---
tags:
- text classification
- clustering
- newsgroups
license: unknown
size: 70 MB
language:
- en
description: >
  The 20 Newsgroups dataset is a collection of approximately 20,000 newsgroup
  documents, partitioned across 20 different newsgroups. It's widely used for
  text classification and clustering experiments. The dataset offers three
  versions: the original, a date-sorted version, and a version with only "From"
  and "Subject" headers.
homepage: http://qwone.com/~jason/20Newsgroups/
task_categories:
- text-classification
- summarization
- question-answering
---

# 20 Newsgroups Dataset

## Introduction

The 20 Newsgroups dataset comprises roughly 20,000 documents from newsgroups, with an almost even distribution across 20 distinct newsgroups. Initially gathered by Ken Lang, this dataset has gained prominence in the machine learning community, particularly for text-related applications like classification and clustering.

## Dataset Structure

The dataset's organization is based on 20 different newsgroups, each representing a unique topic. While some of these newsgroups share similarities or are closely related, others are quite distinct from one another.

### List of Newsgroups:

- Computer Graphics
- Windows OS Miscellaneous
- IBM PC Hardware
- Mac Hardware
- Windows X
- Automobiles
- Motorcycles
- Baseball
- Hockey
- Cryptography
- Electronics
- Medicine
- Space
- Miscellaneous Sales
- Miscellaneous Politics
- Politics & Guns
- Middle East Politics
- Miscellaneous Religion
- Atheism
- Christianity

## Sample Entries

### Sample from `Windows X`

```
From: Bill.Kayser@delft.SGp.slb.COM (Bill Kayser)
Subject: Re: TeleUse, UIM/X, and C++
Article-I.D.: parsival.199304060629.AA00339
Organization: The Internet
Lines: 25
NNTP-Posting-Host: enterpoop.mit.edu
To: xpert@expo.lcs.mit.edu
Cc: Bill.Kayser@delft.sgp.slb.com


> 
> Does anyone have any good ideas on how to integrate C++ code elegantly
> with TeleUse, UIM/X / Interface Architect generated code?
> 
> Source would be great, but any suggestions are welcome.

It's my understanding that the next release of UIM/X, due out
last February :-) has full support for C++.

I use XDesigner which does not have the interpreter or UI meta languages
of these other tools but does fully support C++ code generation,
reusable templates via C++ classes which are generated, a variety of
other handy features for using C++ and layout functions in different
ways, and generates Motif 1.2 code (including drag 'n drop, 
internationalization, etc.).  Fits in quite nicely with Doug Young's
paradigm for C++/Motif.

Available in the US from VI Corp, in Europe from Imperial Software,
London (see FAQ for details).

Bill
________________________________________________________________________
                                           Schlumberger Geco Prakla
                                           kayser@delft.sgp.slb.com

```


### Sample from `Electronics`

```
From: baden@sys6626.bison.mb.ca (baden de bari)
Subject: Re: Jacob's Ladder
Organization: System 6626 BBS, Winnipeg Manitoba Canada
Lines: 36

g92m3062@alpha.ru.ac.za (Brad Meier) writes:

> Hi, I'm looking for a circuit, that is called a "Jacob's Ladder".
> This little box is usually seen in sci-fi movies.  It consists of 
> two curves of wire protruding into the air, with little blue sparks
> starting at their base (where the two wires are closer to each other),
> moving up the wires to the top, and ending in a small crackling noise. 
> 
> Could anyone supply me with the schematic for the innards of this box?
> 
> Thanks in advance
> Mike
> 
> (Please reply by email to g90k3853@alpha.ru.ac.za)
> 
> --
>  |  / |  | ~|~ /~~\  |  | ~|~ /~~\  |~~\  /~~\         The KnightOrc 
>  |/   |\ |  | |   __ |__|  | |    | |__/ |         g92m3062@hippo.ru.ac.za
>  |\   | \|  | |    | |  |  | |    | |  | |      "When it's over I'll go home,
>  |  \ |  | _|_ \__/  |  |  |  \__/  |  |  \__/    until then, I stay!" - Me

 
        I'd like any accumulated information on this as well please.
 
                Thanks.
 
_________________________________________    
    _____                                |
   |     |   |                           |
 =========== |    Baden de Bari          |
  | o   o |  |                           |
  |   ^   |  | baden@sys6626.bison.ca    |
  |  {-}  |  | baden@inqmind.bison.ca    |
   \_____/   |                           |
-----------------------------------------

```

## Data Availability

The dataset is bundled in `.tar.gz` format. Within each bundle, individual subdirectories represent a newsgroup. Every file within these subdirectories corresponds to a document posted in that specific newsgroup.

There are three primary versions of the dataset:

1. The original version, which remains unaltered.
2. A version sorted by date, which segregates the data into training (60%) and test (40%) sets. This version has removed duplicates and some headers for clarity.
3. A version that only retains the "From" and "Subject" headers, with duplicates removed.

For those seeking a more consistent benchmark, the date-sorted version is recommended. It offers a realistic split based on time and has removed any newsgroup-specific identifiers.

## Matlab/Octave Version

For users of Matlab or Octave, a processed variant of the date-sorted dataset is available. This version is structured as a sparse matrix and includes files like `train.data`, `train.label`, `test.data`, and more. Additionally, a vocabulary file is provided to help users understand the indexed data.

## Additional Information

For more details and the original dataset, you can refer to the [official website](http://qwone.com/~jason/20Newsgroups/).



---
license: cc-by-nc-4.0
---