<jupyter_start><jupyter_text>Convert table of questions to MarkdownThis notebook can be use to transform a CSV file containing questions and answers to create a markdown file. The table should have four columns: ```Category name```, ```Question```, ```Answer```, ```Similar questions```. The questions and answers will be grouped according to their category. If a several questions are similar, the additional questions can be added in the ```Similar questions``` columns, separated by ```; ``` (don't forget the space after ```;```).To generate the CVS file, you can just export a Google Sheet using File->Download->Comma separated. An example of such a file can be found [here](https://docs.google.com/spreadsheets/d/1aL77bMOsQeYoWBA8_jZx_mu13KeL_bgJHNVMOWPHHHA/edit?usp=sharing).<jupyter_code>import pandas as pd
import numpy as np
from IPython.display import display, Markdown, Latex, HTML
# import CSV. Use the appropriate path
questions = pd.read_csv('questions.csv')

# group questions by topic 
groups = questions.groupby('Category name')
# creata a list of topics that should be ordered
ordered_topics = ['Category3', 'Category1']

# add any missing topic at the end of the list
final_list = ordered_topics + [x for x in questions['Category name'].dropna().unique() if x not in ordered_topics]

# keep only headers that are actually used
final_list = [f for f in final_list if f in groups.groups.keys()]
# add a specific suffix to tags, in case the same headers are used in multiple posts
suffix = '_2'
# create content
mdstring = '## Table of contents  \n'
for g in final_list:
    mdstring +="* <a href=\"#" + g.lower() + suffix + "\">" + g + "</a>  \n"

mdstring += '  \n'
for g in final_list:
    
    g2 = groups.get_group(g)
    mdstring +="## <a name=\"" + g.lower()+ suffix + "\" /></a>"+ g +"  \n"
        
    for q in range(len(g2)):
        if g2.iloc[q]['Question'] is not None and g2.iloc[q]['Answer'] is not None and g2.iloc[q]['Answer'] is not np.nan:
            mdstring = mdstring + '**' + g2.iloc[q]['Question'] + '**  \n  \n'
            
            if not pd.isnull(g2.iloc[q]['Similar questions']):
                additional_q = g2.iloc[q]['Similar questions']
                additional_q = additional_q.split('; ')
                for ad in additional_q:
                    mdstring = mdstring + '**' + ad + '**  \n  \n'
                
            mdstring = mdstring + g2.iloc[q]['Answer'] + '  \n  \n'
        
display(Markdown(mdstring))
with open('QA.md','w+') as f:
    f.writelines(mdstring)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13406079501169135, 'fraction_numerical': 0.007014809041309431, 'mean_word_length': 3.3361486486486487, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 11, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '569210', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 822, 'n_tokens_neox': 796, 'n_words': 283, 'path': 'utils/questions_to_markdown.ipynb', 'repo_name': 'guiwitz/neubias_academy_biapy', 'size': '2566'}