region:us
The rapid progress of large language models has led to impressive results in a wide array of tasks. However, there remains a need for increasingly challenging datasets to evaluate these models' ability to handle long-range dependencies. In this paper, we present Pathfinder-X2, a novel dataset that builds upon the Pathfinder and Pathfinder-X datasets. Pathfinder-X2 comprises 512x512 pixel images, designed to test large language models' capacity to segment a specific white line dash "snake" with a circle at its tip among a collection of similar, distractor snakes. The increased image resolution and complexity of Pathfinder-X2 present a substantially more challenging task for large language models, contributing to the ongoing development and assessment of such models.
@article{suard2023pathfinder,
  title={Pathfinder-X2: A Challenging Dataset for Evaluating Large Language Models on Long-Range Dependencies},
  author={Suard, Tyler},
  journal={},
  year={2023}
}
# Pathfinder-X2 

license: CC BY 4.0, Free to use for any purpose, including commercial, with attribution.

The Pathfinder and Pathfinder-X datasets have been crucial for training Large Language Models with Long-Range Dependencies.  
In January of 2023, Meta's Mega LLM scored a 97% on the Pathfinder-X dataset, indicating a need for an even more challenging benchmark.
Pathfinder-X2 contains 200,000 512x512 images along with 200,000 segmentation masks for those images.  Each image contains an assortment
of dashed-line "snakes" of varying length, and a model's task is to segment only the snake with a circle on one end.
Each image is meant to be fed in as a sequence,pixel-by-pixel, into a Large Language Model.

Explanation paper: https://www.overleaf.com/read/rpsmdnxbdfjt

Based on the Pathfinder dataset by Drew Linsley, Alekh K Ashok, Lakshmi N Govindarajan, Rex Liu, and Thomas Serre.

![Sample input](https://huggingface.co/datasets/Tylersuard/PathfinderX2/blob/main/sample_0.png)
![Sample label](https://huggingface.co/datasets/Tylersuard/PathfinderX2/blob/main/seg_sample_0.png)

---
annotations_creators:
- Tyler Suard
tags:
- language
- nlp
- llm
- long-range
size_categories:
- 100K<n<1M
---