<jupyter_start><jupyter_text>We need to get from mapped bam file to single fast5s:Input is:* Genome sequence Pgt21-0* Fastq files to map with minimap. These were combined with "find * -type f -name '*.gz'| xargs -I X cat X >> 20210406.PgtGerminatedSpores.fastq.gz"* Folder with multifast5 of fastqs mapped to the reference. These were combined with "find * -type f -name "*.fast5" | xargs -I X mv X $dir/." on the command lineOutput:* folder with single fast5s that are mappedWhat we need to do:* map all reads* get ID of all mapped fastqs* extract them in batches with the fast5_subset<jupyter_code>import os<jupyter_output><empty_output><jupyter_text>INPUTS  *****Define directories before running on different sampes*****<jupyter_code>###define input directories here 
RefFn = '../../../resources/genomic_resources/chr_A_B_unassigned.fasta'
FastqFn = '../../../analysis/germinatedSpores/20210406.PgtGerminatedSpores.fastq.gz'
BamFn = '../../../analysis/germinatedSpores/20210406.Pgt210.PgtGerminatedSpores.sorted.bam'
InFast5Dir = '/media/ssd-01/ben/projects/Pgt210Methylation/rawData/germinatedSpores/fast5/combined'
OutDir = '../../../analysis/germinatedSpores/subsetFast5'
OutDirGuppy = '../../../analysis/germinatedSpores/subsetFast5/basecalled'<jupyter_output><empty_output><jupyter_text>*****Define directories before running on different sampes*****<jupyter_code>RefFn = os.path.abspath(RefFn)
FastqFn = os.path.abspath(FastqFn)
BamFn = os.path.abspath(BamFn)
InFast5Dir = os.path.abspath(InFast5Dir)
OutDir = os.path.abspath(OutDir)
OutDirGuppy = os.path.abspath(OutDirGuppy)
threads = 8
##map all the reads with Minimap
!minimap2 -2 --sam-hit-only --secondary=no -t {threads} -ax map-ont {RefFn} {FastqFn} | samtools sort -@ {12-threads} -o {BamFn} -O bam -
!samtools index {BamFn}
##generated the read ID list
MappedIdsFn = BamFn.replace('.bam', '.mappedids.txt')
SamtoolsStatsFn = BamFn.replace('.bam', '.stats.txt')
!samtools stats -@ {threads} {BamFn} > {SamtoolsStatsFn}
!samtools  view -F 4  {BamFn} | cut -f 1 | sort | uniq > {MappedIdsFn}
!wc {MappedIdsFn}
#subset the reads
!fast5_subset -i {InFast5Dir} -s {OutDir} -l {MappedIdsFn} -t {threads + 4}
#basecall the mapped reads only
!guppy_basecaller -i {OutDir} -s {OutDirGuppy} --fast5_out -r --compress_fastq --device auto -c dna_r9.4.1_450bps_hac.cfg
print('hello')<jupyter_output>hello
{'fraction_non_alphanumeric': 0.1423728813559322, 'fraction_numerical': 0.026694915254237287, 'mean_word_length': 4.947103274559194, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 16, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '425831', 'license': 'MIT', 'n_tokens_mistral': 923, 'n_tokens_neox': 873, 'n_words': 219, 'path': 'multi_to_single_fast5_mapped_ids/RevisionGerminatedSporesProcessing.ipynb', 'repo_name': 'Team-Schwessinger/Pgt210Methylation', 'size': '2360'}