<commit_before>===============
UNIX WIZARDRY
===============

The explicit goal of this tutorial is to provide you with some tricks and tools and to make your unix life happier. Also, to possibly introduce you to samtools. 

|

See https://github.com/stephenturner/oneliners for a bunch of examples. 

|

**LAUNCH AN INSTANCE**: Get something small, for instance ``t2.small`` and update it.

::

  sudo apt-get update && sudo apt-get -y upgrade
  sudo apt-get -y install libncurses5-dev build-essential tmux git gcc make g++ python-dev unzip default-jre zlib1g-dev

**ALIASES**: These are 'shortcuts' for make doing stuff more efficient.

::

  alias c='clear'
  alias gh='history | grep'
  alias ll='logout'
  alias l='ls -lth'
  alias mv='mv -i'
  alias cp='cp -i' 
  alias targz='tar -zcf'
  alias utargz='tar -zxf'

**INSTALL TRINITY**: note the ``-j4``

::

  git clone https://github.com/trinityrnaseq/trinityrnaseq.git
  cd trinityrnaseq
  make -j4
  PATH=$PATH:$(pwd)

**.profile customization**

::

  nano ~/.profile

**DOWNLOAD SOME DATA**

::

  mkdir /home/ubuntu/data
  cd /home/ubuntu/data
  curl -LO https://www.dropbox.com/s/5fymuyb1f2l8kfj/ngsfile.tar.gz
  utargz ngsfile.tar.gz


**Find a file**

::

  cd $HOME
  find / -name Trinity.fasta
  find / -type f -size +10M 2> /dev/null

**sed**

::

  cd /home/ubuntu/data/ngs2015
  less Trinity.fasta
  sed 's_|_-_g' Trinity.fasta | grep ^'>' | head
  sed -i 's_|_-_g' Trinity.fasta
  
**Number fasta def line**

::

  awk '/^>/{print ">" ++i; next}{print}' < Trinity.fasta > Trinity.numbered.fasta
  

**awk**

::

  less Trinity.counts.RNAseq.txt
  awk '{print $1 "\t" $3}' Trinity.counts.RNAseq.txt | head
  awk '$1 == "c996_g1_i1"' Trinity.counts.RNAseq.txt

**Random Stuff**

::

  cd -
  tmux/screen

========================
SAMTOOLS
========================

Let's learn something about samtools

**INSTALL**

::

  cd $HOME
  git clone https://github.com/samtools/htslib.git
  cd htslib && make -j4 && cd ../
  git clone https://github.com/samtools/samtools.git
  cd samtools && make -j4 && PATH=$PATH:$(pwd) && cd ../


**EXPLORE**<commit_msg>tmux aliases<commit_after>===============
UNIX WIZARDRY
===============

The explicit goal of this tutorial is to provide you with some tricks and tools and to make your unix life happier. Also, to possibly introduce you to samtools. 

|

See https://github.com/stephenturner/oneliners for a bunch of examples. 

|

**LAUNCH AN INSTANCE**: Get something small, for instance ``t2.small`` and update it.

::

  sudo apt-get update && sudo apt-get -y upgrade
  sudo apt-get -y install libncurses5-dev build-essential tmux git gcc make g++ python-dev unzip default-jre zlib1g-dev

**ALIASES**: These are 'shortcuts' for make doing stuff more efficient.

::

  alias c='clear'
  alias gh='history | grep'
  alias ll='logout'
  alias l='ls -lth'
  alias mv='mv -i'
  alias cp='cp -i' 
  alias targz='tar -zcf'
  alias utargz='tar -zxf'
  
  alias tn="tmux new -s"
  alias ta="tmux attach -t"
  alias tl="tmux ls"
  alias tk="tmux kill-session -t"

**INSTALL TRINITY**: note the ``-j4``

::

  git clone https://github.com/trinityrnaseq/trinityrnaseq.git
  cd trinityrnaseq
  make -j4
  PATH=$PATH:$(pwd)

**.profile customization**

::

  nano ~/.profile

**DOWNLOAD SOME DATA**

::

  mkdir /home/ubuntu/data
  cd /home/ubuntu/data
  curl -LO https://www.dropbox.com/s/5fymuyb1f2l8kfj/ngsfile.tar.gz
  utargz ngsfile.tar.gz


**Find a file**

::

  cd $HOME
  find / -name Trinity.fasta
  find / -type f -size +10M 2> /dev/null

**sed**

::

  cd /home/ubuntu/data/ngs2015
  less Trinity.fasta
  sed 's_|_-_g' Trinity.fasta | grep ^'>' | head
  sed -i 's_|_-_g' Trinity.fasta
  
**Number fasta def line**

::

  awk '/^>/{print ">" ++i; next}{print}' < Trinity.fasta > Trinity.numbered.fasta
  

**awk**

::

  less Trinity.counts.RNAseq.txt
  awk '{print $1 "\t" $3}' Trinity.counts.RNAseq.txt | head
  awk '$1 == "c996_g1_i1"' Trinity.counts.RNAseq.txt

**Random Stuff**

::

  cd -
  tmux/screen


========================
SAMTOOLS
========================

Let's learn something about samtools

**INSTALL**

::

  cd $HOME
  git clone https://github.com/samtools/htslib.git
  cd htslib && make -j4 && cd ../
  git clone https://github.com/samtools/samtools.git
  cd samtools && make -j4 && PATH=$PATH:$(pwd) && cd ../


**EXPLORE**
{'fraction_non_alphanumeric': 0.18027132674177973, 'fraction_numerical': 0.011956771671648655, 'mean_word_length': 3.319761668321748, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 13, 'https://': 10, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7079546', 'n_tokens_mistral': 1757, 'n_tokens_neox': 1616, 'n_words': 451}