<jupyter_start><jupyter_text>Generating a Segmentation Mesh  with NeuroglancerThis module demonstrates how to generate a mesh from a CloudVolume segmentation layer.<jupyter_code># We need to add balsam and the modules it depends on to the Python search paths. 
import sys
sys.path.insert(0,'/soft/datascience/Balsam/0.3.5.1/env/lib/python3.6/site-packages/')
sys.path.insert(0,'/soft/datascience/Balsam/0.3.5.1/')

# We also need balsam and postgresql to be in the path. (Misha suggests this may not be necessary)
import os
os.environ['PATH'] ='/soft/datascience/Balsam/0.3.5.1/env/bin/:' + os.environ['PATH']
os.environ['PATH'] +=':/soft/datascience/PostgreSQL/9.6.12/bin/'
try:
    import balsam
except ImportError:
    print('Cannot find balsam, make sure balsam is installed or it is available in Python search paths')
    
os.environ["BALSAM_DB_PATH"]='/lus/theta-fs0/projects/connectomics_aesp/balsam_database/'

from balsam_helper import *

# Import widgets
from ipywidgets import interact, interactive
from ipywidgets import fixed, interact_manual 
from ipywidgets import Textarea, widgets, Layout, Accordion
from ipywidgets import VBox, HBox, Box, Text, BoundedIntText

env_preamble = '/lus/theta-fs0/projects/connectomics_aesp/software/HappyNeuron/macros_theta/theta_build_preamble.sh'<jupyter_output><empty_output><jupyter_text>Set up the ApplicationBefore submitting image conversion jobs, make sure there is a Balsam application available to you. Here, we set up a HappyNeuron application and meshing workflow. The application will run the mesh_generator executable installed with HappyNeuron.<jupyter_code>add_app(
    'HappyNeuron_meshing',
    'python /home/kinnison/HappyNeuron/happyneuron/mesh/mesh_generator.py',  # 'mesh_generator',
    description='Create a 3D segmentation mesh.',
    envscript=env_preamble
)<jupyter_output><empty_output><jupyter_text>Set up the Workflow and JobThis is where job parameters will be set and added to a workflow in the application.<jupyter_code>seg_path = '/lus/theta-fs0/projects/connectomics_aesp/pipeline_data/generate_mesh/precomputed/segmentation'

args = f'--labels {seg_path}'

job_id = add_job(
    'generate_mesh',  # Job Name
    'generate_mesh_rafcube',   # Workflow Name
    'HappyNeuron_meshing',
    description='Convert the Allen stack to a CloudVolume image layer.',
    args=args,
    num_nodes=2,
    ranks_per_node=4
)<jupyter_output><empty_output><jupyter_text>Run the Conversion<jupyter_code>submit(project='connectomics_aesp',
   queue='debug-flat-quad',
   nodes=2,
   wall_minutes=20,
   wf_filter='generate_mesh_rafcube'
)
def get_job_info(job_id='',show_output=False):
    """
    Prints verbose job info for a given job id.
    Parameters
    ----------
    job_id: str, Partial or full Balsam job id.
    """
    from balsam.launcher.dag import BalsamJob as Job
    jobs = Job.objects.all().filter(job_id__contains=job_id)
    if len(jobs) == 1:
        thejob = jobs[0]
        print(jobs[0])
        if show_output:
            output = f'{thejob.working_directory}/{thejob.name}.out'
            with open(output) as f:
                out = f.read()
            print(f'Output file {output} content:')
            print(out)
    elif len(jobs) == 0:
        print('No matching jobs')
    else:
        print(f'{len(jobs)} jobs matched, enter full id.')
        
get_job_info(job_id=job_id, show_output=True)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10601635514018691, 'fraction_numerical': 0.009637850467289719, 'mean_word_length': 4.2051671732522795, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 3, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '92038', 'license': 'BSD-Source-Code', 'n_tokens_mistral': 1178, 'n_tokens_neox': 1133, 'n_words': 281, 'path': 'notebooks_theta/generate_mesh.ipynb', 'repo_name': 'jeffkinnison/HappyNeuron', 'size': '3424'}