<jupyter_start><jupyter_text>Access via ssh and gateway to the Google Colab virtual machine Installing the ssh client on the local computer Windows 10upgrade to at least 'autumn 2017'install the "OpenSSH client" functionhttps://pureinfotech.com/install-openssh-client-windows-10/ Linux/macOSThe ssh client is built into the system  SFTP client installation (for file transfer)Install FileZilla clienthttps://filezilla-project.org Configuring terminal tools in a virtual machine  Installation and launch of the ssh serve https://help.ubuntu.com/community/SSHIntroductionSSH ("Secure SHell") is a protocol for securely accessing one computer from another. Despite the name, SSH allows you to run command line and graphical programs, transfer files, and even create secure virtual private networks over the Internet.To use SSH, you will need to install an SSH client on the computer you connect from, and an SSH server on the computer you connect to. The most popular Linux SSH client and Linux SSH server are maintained by the OpenSSH project.The OpenSSH client is included in Ubuntu by default. For information on connecting to an SSH server, see Connecting to an OpenSSH Server.To install the OpenSSH server, install the following package: openssh-server.<jupyter_code>!apt-get -qq update
!apt-get -qq install -y openssh-server
fn_sshd_config = '/etc/ssh/sshd_config'

sshd_config = '''
PermitRootLogin yes
PubkeyAuthentication yes
AuthorizedKeysFile      %h/.ssh/authorized_keys
UsePAM yes
X11Forwarding yes
PrintMotd no
AcceptEnv LANG LC_*
Subsystem	sftp	/usr/lib/openssh/sftp-server
'''

!echo "$sshd_config" > $fn_sshd_config
!/etc/init.d/ssh restart
!/etc/init.d/ssh status<jupyter_output><empty_output><jupyter_text>Nvidia-smi configuration<jupyter_code>fn_bashrc = '/root/.bashrc'

bashrc = '''
export PATH="$PATH:/opt/bin"
export LD_LIBRARY_PATH="/usr/lib64-nvidia:$LD_LIBRARY_PATH"
'''

!echo "$bashrc" >> $fn_bashrc
# watch -n1 nvidia-smi<jupyter_output><empty_output><jupyter_text>Installing tmux, mc, htop, glances<jupyter_code>!apt-get -qq -y install tmux mc htop 
!pip install -q glances<jupyter_output><empty_output><jupyter_text>Configuration and launching the gateway on portmap.iohttps://portmap.io/  Creating an account and generating a private key public key authenticationWith public key authentication, the authenticating entity has a public key and a private key. Each key is a large number with special mathematical properties. The private key is kept on the computer you log in from, while the public key is stored on the .ssh/authorized_keys file on all the computers you want to log in to. When you log in to a computer, the SSH server uses the public key to "lock" messages in a way that can only be "unlocked" by your private key - this means that even the most resourceful attacker can't snoop on, or interfere with, your session. public key format:   @1. create and activate account2. add configuration3. generate and download private key4. create and activate 'mapping rule'  Export of private key from public on the local computer```ssh-keygen -y -f private_key.pem > public_key.pub```  Launching the gateway on portmap.ioIf you already have an account set up at portmap.io, then:1. Sign In2. go to 'mapping rules'3. click edit4. click update  Configuring root access on a virtual machine<jupyter_code>!mkdir -p /root/.ssh
!chmod 700 /root/.ssh


fn_authorized_keys = '/root/.ssh/authorized_keys'

public_key = '''
ssh-rsa <KEY>
'''

!echo "$public_key" > $fn_authorized_keys
!chmod 600 $fn_authorized_keys
ls -la /root/.ssh<jupyter_output><empty_output><jupyter_text>Configuring the ssh tunnel from the virtual machine to portmap.io<jupyter_code>fn_private_key_pem = '/root/.ssh/private_key.pem'

private_key_pem = '''
-----<KEY>
'''

!echo "$private_key_pem" > $fn_private_key_pem
!chmod 600 $fn_private_key_pem
ls -l /root/.ssh
# ssh tunnel to portmap.io
# For remote forwarding: GatewayPorts is always turned on on the server side, 
# and so all remote forwardings are accessible from anywhere.
# Here localhost:22 is the tcp port of the colab VM machine, while moonjung_*.portmap.io is the remote ssh-server
#?? https://askubuntu.com/questions/50064/reverse-port-tunnelling
# ssh -i ~/.ssh/moonjung.first.pem moonjung.first@moonjung-42209.portmap.io -N -R 42209:localhost:22
options = '-oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null'
!ssh $options -i $fn_private_key_pem -f -R  42209:localhost:22 <EMAIL>.<EMAIL>@<EMAIL>-42209.portmap.io -N
#ssh tunnel monitoring
!ps ax|grep ssh
# killing the ssh process
!kill 1739<jupyter_output><empty_output><jupyter_text>Work Log on to the virtual machine from the local computer through the gateway  portmap.io gate: moonjung-53525.portmap.iogateway port: 42209user: rootauthentication: private_key.pem-L [protocol/][listening_host:] listening_port:host:hostport When a Secure Shell connection is established, the **Secure Shell client** opens a socket on the Secure Shell client host using the designated local port (listening_port). (On client hosts with multiple interfaces, use listening_host to specify which interface.) Configure your application client (the one whose data you want to forward) to send data to the forwarded socket (rather than directly to the destination host and port). When that client establishes a connection, all data sent to the forwarded port is redirected through the secure tunnel to the **Secure Shell server**, which decrypts it and then directs it to the destination socket (host,hostport). Unless the gateway ports option is enabled, the forwarded local port is available only to clients running on the same computer as the Secure Shell client. The optional protocol can be tcp or ftp. Multiple client applications can use the forwarded port, but the forward is active only while ssh is running.-R [protocol/][listening_host:] listening_port:host:hostportWhen the Secure Shell connection is established, the **Secure Shell server** opens a socket on its host (the computer running the Secure Shell server) using the designated remote port (listening_port). (On server hosts with multiple interfaces, use listening_host to specify which interface.) Configure your client application (the one whose data you want to forward) to send data to the forwarded socket (rather than directly to the destination host and port). When that client establishes a connection, all data sent to the forwarded port is redirected through the secure tunnel to the **Secure Shell client**, which decrypts it and then directs it to the destination socket (host,hostport). The optional protocol can be tcp or ftp.forwarding port 8890 from the virtual machine to the local computer: -L 8890:localhost:8890 =>> from the local computer to the virtual machine?option: -p port Connect to this port. Server must be on the same port.==Port to connect to on the remote host```ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -i ~/.ssh/private_key.pem -p 42209 -L 8890:localhost:8890 <EMAIL>.portmap.io ```use flag -f -N if you only want port forwarding  Multiplex terminalnew session```tmux```join an existing session```tmux at```tmux cheat sheethttps://gist.github.com/russelldb/06873e0ad4f5ba1c4eec1b673ff4d4cd Manage Files```mc```  Monitor RAM, CPU, processes```glances```  Monitor GPU```watch -n1 nvidia-smi```  Upload filesbetween the local computer and virtual machine```FileZilla```  Monitor training progress```python -m visdom.server -p 8890```    http://localhost:8890```tensorboard --logdir ...```<jupyter_code>!uptime<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07930214115781126, 'fraction_numerical': 0.016521279407877346, 'mean_word_length': 5.274461028192372, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 39, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '604959', 'license': 'MIT', 'n_tokens_mistral': 2336, 'n_tokens_neox': 2109, 'n_words': 918, 'path': 'Copy_of_demo_colab_ssh_access.ipynb', 'repo_name': 'moonryul/DeepLearningMaster2019', 'size': '9613'}