<jupyter_start><jupyter_text>Phase 1: have the transformers example running<jupyter_code>def hello_world_tranformers_example():
    tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
    model = TFAutoModel.from_pretrained("bert-base-uncased")
    inputs = tokenizer("Hello world!", return_tensors="tf")
    outputs = model(**inputs)
    print(outputs)<jupyter_output><empty_output><jupyter_text>Phase 2: modify the transformers example to run in Chinese<jupyter_code>def chinese_transformers_example():
    tokenizer = AutoTokenizer.from_pretrained("bert-base-multilingual-uncased")
    model = TFAutoModel.from_pretrained("bert-base-multilingual-uncased")
    inputs = tokenizer("你好!", return_tensors="tf")
    outputs = model(**inputs)
    print(outputs)<jupyter_output><empty_output><jupyter_text>Phase 3: modify the transformers example to run as a text generator (English first)<jupyter_code>def text_generation_with_bert_example():
    sentence_fuser = EncoderDecoderModel.from_pretrained("google/roberta2roberta_L-24_discofuse")
    tokenizer = AutoTokenizer.from_pretrained("google/roberta2roberta_L-24_discofuse")
    summary_text = 'This is the first sentence.'
    input_ids = tokenizer(summary_text, add_special_tokens=False, return_tensors="pt").input_ids
    outputs = sentence_fuser.generate(input_ids)
    print('Generated {cnt} pieces.'.format(cnt=len(outputs)))
    print(tokenizer.decode(outputs[0]))<jupyter_output><empty_output><jupyter_text>Phase 4: train text generator with custom text corpus  Phase 5: train text generator with Chinese text corpus  Phase 6: use Chinese in the text generator TODO(tianhaoz95): think about the next steps to build a comment generator  The main program to test things out<jupyter_code># hello_world_tranformers_example()
# chinese_transformers_example()
text_generation_with_bert_example()<jupyter_output>Generated 1 pieces.
This is the first sentence. In fact, the is
{'fraction_non_alphanumeric': 0.09550851832731028, 'fraction_numerical': 0.008260196179659268, 'mean_word_length': 6.9753086419753085, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 16, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '93181', 'license': 'MIT', 'n_tokens_mistral': 624, 'n_tokens_neox': 598, 'n_words': 130, 'path': 'example.ipynb', 'repo_name': 'CommentGen/experimental', 'size': '1937'}