Data SelectionData selection for fine-tuning has been extensively studied, focusing on improving quality (Li et al., 2023), diversity (Liu et al., 2024), and distribution matching (Li et al., 2023; Xia et al., 2024; Ni et al., 2024). For pretraining, various lightweight filters are utilized (Albalak et al., 2024), including heuristic-based (_e.g.,_ language and item count filtering), classifier-based (Brown et al., 2020), and perplexity-based approaches (Wenzek et al., 2019). The massive public RedPajama-Data-v2 dataset (Computer, 2023), for example, leverages over 40 quality indicators for data filtering and reweighting. Nevertheless, strict filtering like blocklist (Raffel et al., 2020) and Safety API filtering (Welbl et al., 2021), have been found to hurt evaluation loss or induce bias (Dodge et al., 2021). To our knowledge, we are the first to explore token-level data selection, aimed at enhancing data quality and information density at the most fundamental granularity.