.

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 coding-pot
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

LICENSE-CreativeML

@@ -0,0 +1,82 @@
+Copyright (c) 2022 Robin Rombach and Patrick Esser and contributors
+
+CreativeML Open RAIL-M
+dated August 22, 2022
+
+Section I: PREAMBLE
+
+Multimodal generative models are being widely adopted and used, and have the potential to transform the way artists, among other individuals, conceive and benefit from AI or ML technologies as a tool for content creation.
+
+Notwithstanding the current and potential benefits that these artifacts can bring to society at large, there are also concerns about potential misuses of them, either due to their technical limitations or ethical considerations.
+
+In short, this license strives for both the open and responsible downstream use of the accompanying model. When it comes to the open character, we took inspiration from open source permissive licenses regarding the grant of IP rights. Referring to the downstream responsible use, we added use-based restrictions not permitting the use of the Model in very specific scenarios, in order for the licensor to be able to enforce the license in case potential misuses of the Model may occur. At the same time, we strive to promote open and responsible research on generative models for art and content generation.
+
+Even though downstream derivative versions of the model could be released under different licensing terms, the latter will always have to include - at minimum - the same use-based restrictions as the ones in the original license (this license). We believe in the intersection between open and responsible AI development; thus, this License aims to strike a balance between both in order to enable responsible open-science in the field of AI.
+
+This License governs the use of the model (and its derivatives) and is informed by the model card associated with the model.
+
+NOW THEREFORE, You and Licensor agree as follows:
+
+1. Definitions
+
+- "License" means the terms and conditions for use, reproduction, and Distribution as defined in this document.
+- "Data" means a collection of information and/or content extracted from the dataset used with the Model, including to train, pretrain, or otherwise evaluate the Model. The Data is not licensed under this License.
+- "Output" means the results of operating a Model as embodied in informational content resulting therefrom.
+- "Model" means any accompanying machine-learning based assemblies (including checkpoints), consisting of learnt weights, parameters (including optimizer states), corresponding to the model architecture as embodied in the Complementary Material, that have been trained or tuned, in whole or in part on the Data, using the Complementary Material.
+- "Derivatives of the Model" means all modifications to the Model, works based on the Model, or any other model which is created or initialized by transfer of patterns of the weights, parameters, activations or output of the Model, to the other model, in order to cause the other model to perform similarly to the Model, including - but not limited to - distillation methods entailing the use of intermediate data representations or methods based on the generation of synthetic data by the Model for training the other model.
+- "Complementary Material" means the accompanying source code and scripts used to define, run, load, benchmark or evaluate the Model, and used to prepare data for training or evaluation, if any. This includes any accompanying documentation, tutorials, examples, etc, if any.
+- "Distribution" means any transmission, reproduction, publication or other sharing of the Model or Derivatives of the Model to a third party, including providing the Model as a hosted service made available by electronic or other remote means - e.g. API-based or web access.
+- "Licensor" means the copyright owner or entity authorized by the copyright owner that is granting the License, including the persons or entities that may have rights in the Model and/or distributing the Model.
+- "You" (or "Your") means an individual or Legal Entity exercising permissions granted by this License and/or making use of the Model for whichever purpose and in any field of use, including usage of the Model in an end-use application - e.g. chatbot, translator, image generator.
+- "Third Parties" means individuals or legal entities that are not under common control with Licensor or You.
+- "Contribution" means any work of authorship, including the original version of the Model and any modifications or additions to that Model or Derivatives of the Model thereof, that is intentionally submitted to Licensor for inclusion in the Model by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Model, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution."
+- "Contributor" means Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Model.
+
+Section II: INTELLECTUAL PROPERTY RIGHTS
+
+Both copyright and patent grants apply to the Model, Derivatives of the Model and Complementary Material. The Model and Derivatives of the Model are subject to additional terms as described in Section III.
+
+2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare, publicly display, publicly perform, sublicense, and distribute the Complementary Material, the Model, and Derivatives of the Model.
+3. Grant of Patent License. Subject to the terms and conditions of this License and where and as applicable, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this paragraph) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Model and the Complementary Material, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Model to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Model and/or Complementary Material or a Contribution incorporated within the Model and/or Complementary Material constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for the Model and/or Work shall terminate as of the date such litigation is asserted or filed.
+
+Section III: CONDITIONS OF USAGE, DISTRIBUTION AND REDISTRIBUTION
+
+4. Distribution and Redistribution. You may host for Third Party remote access purposes (e.g. software-as-a-service), reproduce and distribute copies of the Model or Derivatives of the Model thereof in any medium, with or without modifications, provided that You meet the following conditions:
+Use-based restrictions as referenced in paragraph 5 MUST be included as an enforceable provision by You in any type of legal agreement (e.g. a license) governing the use and/or distribution of the Model or Derivatives of the Model, and You shall give notice to subsequent users You Distribute to, that the Model or Derivatives of the Model are subject to paragraph 5. This provision does not apply to the use of Complementary Material.
+You must give any Third Party recipients of the Model or Derivatives of the Model a copy of this License;
+You must cause any modified files to carry prominent notices stating that You changed the files;
+You must retain all copyright, patent, trademark, and attribution notices excluding those notices that do not pertain to any part of the Model, Derivatives of the Model.
+You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions - respecting paragraph 4.a. - for use, reproduction, or Distribution of Your modifications, or for any such Derivatives of the Model as a whole, provided Your use, reproduction, and Distribution of the Model otherwise complies with the conditions stated in this License.
+5. Use-based restrictions. The restrictions set forth in Attachment A are considered Use-based restrictions. Therefore You cannot use the Model and the Derivatives of the Model for the specified restricted uses. You may use the Model subject to this License, including only for lawful purposes and in accordance with the License. Use may include creating any content with, finetuning, updating, running, training, evaluating and/or reparametrizing the Model. You shall require all of Your users who use the Model or a Derivative of the Model to comply with the terms of this paragraph (paragraph 5).
+6. The Output You Generate. Except as set forth herein, Licensor claims no rights in the Output You generate using the Model. You are accountable for the Output you generate and its subsequent uses. No use of the output can contravene any provision as stated in the License.
+
+Section IV: OTHER PROVISIONS
+
+7. Updates and Runtime Restrictions. To the maximum extent permitted by law, Licensor reserves the right to restrict (remotely or otherwise) usage of the Model in violation of this License, update the Model through electronic means, or modify the Output of the Model based on updates. You shall undertake reasonable efforts to use the latest version of the Model.
+8. Trademarks and related. Nothing in this License permits You to make use of Licensors’ trademarks, trade names, logos or to otherwise suggest endorsement or misrepresent the relationship between the parties; and any rights not expressly granted herein are reserved by the Licensors.
+9. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Model and the Complementary Material (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Model, Derivatives of the Model, and the Complementary Material and assume any risks associated with Your exercise of permissions under this License.
+10. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Model and the Complementary Material (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages.
+11. Accepting Warranty or Additional Liability. While redistributing the Model, Derivatives of the Model and the Complementary Material thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability.
+12. If any provision of this License is held to be invalid, illegal or unenforceable, the remaining provisions shall be unaffected thereby and remain valid as if such provision had not been set forth herein.
+
+END OF TERMS AND CONDITIONS
+
+
+
+
+Attachment A
+
+Use Restrictions
+
+You agree not to use the Model or Derivatives of the Model:
+- In any way that violates any applicable national, federal, state, local or international law or regulation;
+- For the purpose of exploiting, harming or attempting to exploit or harm minors in any way;
+- To generate or disseminate verifiably false information and/or content with the purpose of harming others;
+- To generate or disseminate personal identifiable information that can be used to harm an individual;
+- To defame, disparage or otherwise harass others;
+- For fully automated decision making that adversely impacts an individual’s legal rights or otherwise creates or modifies a binding, enforceable obligation;
+- For any use intended to or which has the effect of discriminating against or harming individuals or groups based on online or offline social behavior or known or predicted personal or personality characteristics;
+- To exploit any of the vulnerabilities of a specific group of persons based on their age, social, physical or mental characteristics, in order to materially distort the behavior of a person pertaining to that group in a manner that causes or is likely to cause that person or another person physical or psychological harm;
+- For any use intended to or which has the effect of discriminating against individuals or groups based on legally protected characteristics or categories;
+- To provide medical advice and medical results interpretation;
+- To generate or disseminate information for the purpose to be used for administration of justice, law enforcement, immigration or asylum processes, such as predicting an individual will commit fraud/crime commitment (e.g. by text profiling, drawing causal relationships between assertions made in documents, indiscriminate and arbitrarily-targeted use).

LICENSE-OFL

@@ -0,0 +1,93 @@
+Copyright 2023 The Lugrasimo Project Authors (https://github.com/docrepair-fonts/lugrasimo-fonts).
+
+This Font Software is licensed under the SIL Open Font License, Version 1.1.
+This license is copied below, and is also available with a FAQ at:
+http://scripts.sil.org/OFL
+
+
+-----------------------------------------------------------
+SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
+-----------------------------------------------------------
+
+PREAMBLE
+The goals of the Open Font License (OFL) are to stimulate worldwide
+development of collaborative font projects, to support the font creation
+efforts of academic and linguistic communities, and to provide a free and
+open framework in which fonts may be shared and improved in partnership
+with others.
+
+The OFL allows the licensed fonts to be used, studied, modified and
+redistributed freely as long as they are not sold by themselves. The
+fonts, including any derivative works, can be bundled, embedded, 
+redistributed and/or sold with any software provided that any reserved
+names are not used by derivative works. The fonts and derivatives,
+however, cannot be released under any other type of license. The
+requirement for fonts to remain under this license does not apply
+to any document created using the fonts or their derivatives.
+
+DEFINITIONS
+"Font Software" refers to the set of files released by the Copyright
+Holder(s) under this license and clearly marked as such. This may
+include source files, build scripts and documentation.
+
+"Reserved Font Name" refers to any names specified as such after the
+copyright statement(s).
+
+"Original Version" refers to the collection of Font Software components as
+distributed by the Copyright Holder(s).
+
+"Modified Version" refers to any derivative made by adding to, deleting,
+or substituting -- in part or in whole -- any of the components of the
+Original Version, by changing formats or by porting the Font Software to a
+new environment.
+
+"Author" refers to any designer, engineer, programmer, technical
+writer or other person who contributed to the Font Software.
+
+PERMISSION & CONDITIONS
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of the Font Software, to use, study, copy, merge, embed, modify,
+redistribute, and sell modified and unmodified copies of the Font
+Software, subject to the following conditions:
+
+1) Neither the Font Software nor any of its individual components,
+in Original or Modified Versions, may be sold by itself.
+
+2) Original or Modified Versions of the Font Software may be bundled,
+redistributed and/or sold with any software, provided that each copy
+contains the above copyright notice and this license. These can be
+included either as stand-alone text files, human-readable headers or
+in the appropriate machine-readable metadata fields within text or
+binary files as long as those fields can be easily viewed by the user.
+
+3) No Modified Version of the Font Software may use the Reserved Font
+Name(s) unless explicit written permission is granted by the corresponding
+Copyright Holder. This restriction only applies to the primary font name as
+presented to the users.
+
+4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
+Software shall not be used to promote, endorse or advertise any
+Modified Version, except to acknowledge the contribution(s) of the
+Copyright Holder(s) and the Author(s) or with their explicit written
+permission.
+
+5) The Font Software, modified or unmodified, in part or in whole,
+must be distributed entirely under this license, and must not be
+distributed under any other license. The requirement for fonts to
+remain under this license does not apply to any document created
+using the Font Software.
+
+TERMINATION
+This license becomes null and void if any of the above conditions are
+not met.
+
+DISCLAIMER
+THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
+OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
+COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
+DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
+OTHER DEALINGS IN THE FONT SOFTWARE.

README.md

@@ -1,13 +1,66 @@
----
-title: Zero2story
-emoji: 💻
-colorFrom: indigo
-colorTo: green
-sdk: gradio
-sdk_version: 3.46.0
-app_file: app.py
-pinned: false
-license: mit
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Zero2Story
+
+![](assets/overview.png)
+
+Zero2Story is a framework built on top of [PaLM API](https://developers.generativeai.google), [Stable Diffusion](https://en.wikipedia.org/wiki/Stable_Diffusion), [MusicGen](https://audiocraft.metademolab.com/musicgen.html) for ordinary people to create their own stories. This framework consists of the **background setup**, **character setup**, and **interative story generation** phases.
+
+**1. Background setup**: In this phase, users can setup the genre, place, and mood of the story. Especially, genre is the key that others are depending on. 
+
+**2. Character setup**: In this phase, users can setup characters up to four. For each character, users can decide their characteristics and basic information such as name, age, MBTI, and personality. Also, the image of each character could be generated based on the information using Stable Diffusion. 
+- PaLM API translates the given character information into a list of keywords that Stable Diffusion could effectively understands.
+- Then, Stable Diffusion generates images using the keywords as a prompt.
+
+**3. Interactive story generation:**: In this phase, the first few paragraphs are generated solely based on the information from the background and character setup phases. Afterwards, users could choose a direction from the given three options that PaLM API generated. Then, further stories are generated based on users' choice. This cycle of choosing an option and generating further stories are interatively continued until users decides to stop. 
+- In each story generation, users also could generate background images and music that describe each scene using Stable Diffusion and MusicGen.
+- If the generated story, options, image, and music in each turn, users could ask to re-generate them.
+
+## Prerequisites
+
+### PaLM API key
+
+This project heavily depends on [PaLM API](https://developers.generativeai.google). If you want to run it on your own environment, you need to get [PaLM API key](https://developers.generativeai.google/tutorials/setup) and paste it in `.palm_api_key.txt` file within the root directory.
+
+### Packages
+
+Make sure you have installed all of the following prerequisites on your development machine:
+* CUDA Toolkit 11.8 with cuDNN 8 - [Download & Install CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit) It is highly recommended to run on a GPU. If you run it in a CPU environment, it will be very slow.
+* Poetry - [Download & Install Poetry](https://python-poetry.org/docs/#installation) It is the python packaging and dependency manager.
+* SQLite3 v3.37.2 or higher - It is required to be installed due to dependencies.
+    - Ubuntu 22.04 and later
+    ```shell
+    $ sudo apt install libc6 sqlite3 libsqlite3
+    ```
+    - Ubuntu 20.04
+    ```shell
+    $ sudo sh -c 'cat <<EOF >> /etc/apt/sources.list
+      deb http://archive.ubuntu.com/ubuntu/ jammy main
+      deb http://security.ubuntu.com/ubuntu/ jammy-security main
+      EOF'
+    $ sudo apt update
+    $ sudo apt install libc6 sqlite3 libsqlite3
+    ```
+* FFmpeg (Optional) - Installing FFmpeg enables local video mixing, which in turn generates results more quickly than [other methods](https://huggingface.co/spaces/fffiloni/animated-audio-visualizer)
+    ```shell
+    $ sudo apt install ffmpeg
+
+## Run
+
+```shell
+$ poetry install
+$ poetry run python app.py
+```
+
+## Todo
+
+- [ ] Exporting of generated stories as PDF
+
+
+## Stable Diffusion Model Information
+
+### Checkpoints
+- For character image generation: [CIVIT.AI Model 129896](https://civitai.com/models/129896)
+- For background image generation: [CIVIT.AI Model 93931](https://civitai.com/models/93931?modelVersionId=148652)
+
+### VAEs
+- For character image generation: [CIVIT.AI Model 23906](https://civitai.com/models/23906)
+- For background image generation: [CIVIT.AI Model 65728](https://civitai.com/models/65728)

app.py

@@ -1,7 +1,688 @@
+import copy
+import random
+
 import gradio as gr
 
-def greet(name):
-    return "Hello " + name + "!!"
+from constants.css import STYLE
+from constants.init_values import (
+	genres, places, moods, jobs, ages, mbtis, random_names, personalities, default_character_images, styles
+)
+from constants import desc
+
+from interfaces import (
+    ui, chat_ui, story_gen_ui, view_change_ui
+)
+from modules.palmchat import GradioPaLMChatPPManager
+
+with gr.Blocks(css=STYLE) as demo:
+	chat_mode = gr.State("plot_chat")
+
+	chat_state = gr.State({
+		"ppmanager_type": GradioPaLMChatPPManager(),
+		"plot_chat": GradioPaLMChatPPManager(),
+		"story_chat": GradioPaLMChatPPManager(),
+		"export_chat": GradioPaLMChatPPManager(),
+	})
+ 
+	cur_cursor = gr.State(0)
+	cursors = gr.State([])
+
+	gallery_images1 = gr.State(default_character_images)
+	gallery_images2 = gr.State(default_character_images)
+	gallery_images3 = gr.State(default_character_images)
+	gallery_images4 = gr.State(default_character_images)
+
+	with gr.Column(visible=True) as pre_phase:
+		gr.Markdown("# 📖 Zero2Story", elem_classes=["markdown-center"])
+		gr.Markdown(desc.pre_phase_description, elem_classes=["markdown-justify"])
+		pre_to_setup_btn = gr.Button("create a custom story", elem_classes=["wrap", "control-button"])
+
+	with gr.Column(visible=False) as background_setup_phase:
+		gr.Markdown("# 🌐 World setup", elem_classes=["markdown-center"])
+		gr.Markdown(desc.background_setup_phase_description, elem_classes=["markdown-justify"])
+		with gr.Row():
+			with gr.Column():
+				genre_dd = gr.Dropdown(label="genre", choices=genres, value=genres[0], interactive=True, elem_classes=["center-label"])
+			with gr.Column():
+				place_dd = gr.Dropdown(label="place", choices=places["Middle Ages"], value=places["Middle Ages"][0], allow_custom_value=True, interactive=True, elem_classes=["center-label"])
+			with gr.Column():
+				mood_dd = gr.Dropdown(label="mood", choices=moods["Middle Ages"], value=moods["Middle Ages"][0], allow_custom_value=True, interactive=True, elem_classes=["center-label"])
+		
+		with gr.Row():
+			back_to_pre_btn = gr.Button("← back", elem_classes=["wrap", "control-button"], scale=1)
+			world_setup_confirm_btn = gr.Button("character setup →", elem_classes=["wrap", "control-button"], scale=2)
+
+	with gr.Column(visible=False) as character_setup_phase:
+		gr.Markdown("# 👥 Character setup")
+		gr.Markdown(desc.character_setup_phase_description, elem_classes=["markdown-justify"])
+		with gr.Row():
+			with gr.Column():
+				gr.Checkbox(label="character include/enable", value=True, interactive=False)
+				char_gallery1 = gr.Gallery(value=default_character_images, height=256, preview=True)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("name", elem_classes=["markdown-left"], scale=3)
+					name_txt1 = gr.Textbox(random_names[0], elem_classes=["no-label"], scale=3)
+					random_name_btn1 = gr.Button("🗳️", elem_classes=["wrap", "control-button-green", "left-margin"], scale=1)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("age", elem_classes=["markdown-left"], scale=3)
+					age_dd1 = gr.Dropdown(label=None, choices=ages, value=ages[0], elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("mbti", elem_classes=["markdown-left"], scale=3)
+					mbti_dd1 = gr.Dropdown(label=None, choices=mbtis, value=mbtis[0], interactive=True, elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("nature", elem_classes=["markdown-left"], scale=3)
+					personality_dd1 = gr.Dropdown(label=None, choices=personalities, value=personalities[0], interactive=True, elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("job", elem_classes=["markdown-left"], scale=3)
+					job_dd1 = gr.Dropdown(label=None, choices=jobs["Middle Ages"], value=jobs["Middle Ages"][0], allow_custom_value=True, interactive=True, elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"], visible=False):
+					gr.Markdown("style", elem_classes=["markdown-left"], scale=3)
+					creative_dd1 = gr.Dropdown(choices=styles, value=styles[0], allow_custom_value=True, interactive=True, elem_classes=["no-label"], scale=4)
+
+				gen_char_btn1 = gr.Button("gen character", elem_classes=["wrap", "control-button-green"])
+
+			with gr.Column():
+				side_char_enable_ckb1 = gr.Checkbox(label="character include/enable", value=False)
+				char_gallery2 = gr.Gallery(value=default_character_images, height=256, preview=True)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("name", elem_classes=["markdown-left"], scale=3)
+					name_txt2 = gr.Textbox(random_names[1], elem_classes=["no-label"], scale=3)
+					random_name_btn2 = gr.Button("🗳️", elem_classes=["wrap", "control-button-green", "left-margin"], scale=1)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("age", elem_classes=["markdown-left"], scale=3)
+					age_dd2 = gr.Dropdown(label=None, choices=ages, value=ages[1], elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("mbti", elem_classes=["markdown-left"], scale=3)
+					mbti_dd2 = gr.Dropdown(label=None, choices=mbtis, value=mbtis[1], interactive=True, elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("nature", elem_classes=["markdown-left"], scale=3)
+					personality_dd2 = gr.Dropdown(label=None, choices=personalities, value=personalities[1], interactive=True, elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("job", elem_classes=["markdown-left"], scale=3)
+					job_dd2 = gr.Dropdown(label=None, choices=jobs["Middle Ages"], value=jobs["Middle Ages"][1], allow_custom_value=True, interactive=True, elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"], visible=False):
+					gr.Markdown("style", elem_classes=["markdown-left"], scale=3)
+					creative_dd2 = gr.Dropdown(choices=styles, value=styles[0], allow_custom_value=True, interactive=True, elem_classes=["no-label"], scale=4)
+
+				gen_char_btn2 = gr.Button("gen character", elem_classes=["wrap", "control-button-green"])
+
+			with gr.Column():
+				side_char_enable_ckb2 = gr.Checkbox(label="character include/enable", value=False)
+				char_gallery3 = gr.Gallery(value=default_character_images, height=256, preview=True)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("name", elem_classes=["markdown-left"], scale=3)
+					name_txt3 = gr.Textbox(random_names[2], elem_classes=["no-label"], scale=3)
+					random_name_btn3 = gr.Button("🗳️", elem_classes=["wrap", "control-button-green", "left-margin"], scale=1)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("age", elem_classes=["markdown-left"], scale=3)
+					age_dd3 = gr.Dropdown(label=None, choices=ages, value=ages[2], elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("mbti", elem_classes=["markdown-left"], scale=3)
+					mbti_dd3 = gr.Dropdown(label=None, choices=mbtis, value=mbtis[2], interactive=True, elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("nature", elem_classes=["markdown-left"], scale=3)
+					personality_dd3 = gr.Dropdown(label=None, choices=personalities, value=personalities[2], interactive=True, elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("job", elem_classes=["markdown-left"], scale=3)
+					job_dd3 = gr.Dropdown(label=None, choices=jobs["Middle Ages"], value=jobs["Middle Ages"][2], allow_custom_value=True, interactive=True, elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"], visible=False):
+					gr.Markdown("style", elem_classes=["markdown-left"], scale=3)
+					creative_dd3 = gr.Dropdown(choices=styles, value=styles[0], allow_custom_value=True, interactive=True, elem_classes=["no-label"], scale=4)
+
+				gen_char_btn3 = gr.Button("gen character", elem_classes=["wrap", "control-button-green"])
+
+			with gr.Column():
+				side_char_enable_ckb3 = gr.Checkbox(label="character include/enable", value=False)
+				char_gallery4 = gr.Gallery(value=default_character_images, height=256, preview=True)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("name", elem_classes=["markdown-left"], scale=3)
+					name_txt4 = gr.Textbox(random_names[3], elem_classes=["no-label"], scale=3)
+					random_name_btn4 = gr.Button("🗳️", elem_classes=["wrap", "control-button-green", "left-margin"], scale=1)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("age", elem_classes=["markdown-left"], scale=3)
+					age_dd4 = gr.Dropdown(label=None, choices=ages, value=ages[3], elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("mbti", elem_classes=["markdown-left"], scale=3)
+					mbti_dd4 = gr.Dropdown(label=None, choices=mbtis, value=mbtis[3], interactive=True, elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("nature", elem_classes=["markdown-left"], scale=3)
+					personality_dd4 = gr.Dropdown(label=None, choices=personalities, value=personalities[3], interactive=True, elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"]):
+					gr.Markdown("job", elem_classes=["markdown-left"], scale=3)
+					job_dd4 = gr.Dropdown(label=None, choices=jobs["Middle Ages"], value=jobs["Middle Ages"][3], allow_custom_value=True, interactive=True, elem_classes=["no-label"], scale=4)
+
+				with gr.Row(elem_classes=["no-gap"], visible=False):
+					gr.Markdown("style", elem_classes=["markdown-left"], scale=3)
+					creative_dd4 = gr.Dropdown(choices=styles, value=styles[0], allow_custom_value=True, interactive=True, elem_classes=["no-label"], scale=4)
+
+				gen_char_btn4 = gr.Button("gen character", elem_classes=["wrap", "control-button-green"])
+
+		with gr.Row():
+			back_to_background_setup_btn = gr.Button("← back", elem_classes=["wrap", "control-button"], scale=1)
+			character_setup_confirm_btn = gr.Button("generate first stories →", elem_classes=["wrap", "control-button"], scale=2)
+
+	gr.Markdown("### 💡 Plot setup", visible=False)
+	with gr.Accordion("generate chapter titles and each plot", open=False, visible=False) as plot_setup_section:
+		title = gr.Textbox("Title Undetermined Yet", elem_classes=["no-label", "font-big"])
+		# plot = gr.Textbox(lines=10, elem_classes=["no-label", "small-big-textarea"])
+
+		gr.Textbox("Rising action", elem_classes=["no-label"])
+		with gr.Row(elem_classes=["left-margin"]):
+			chapter1_plot = gr.Textbox(placeholder="The plot of the first chapter will be generated here", lines=3, elem_classes=["no-label"])
+
+		gr.Textbox("Crisis", elem_classes=["no-label"])
+		with gr.Row(elem_classes=["left-margin"]):  
+			chapter2_plot = gr.Textbox(placeholder="The plot of the second chapter will be generated here", lines=3, elem_classes=["no-label"])
+
+		gr.Textbox("Climax", elem_classes=["no-label"])
+		with gr.Row(elem_classes=["left-margin"]):  
+			chapter3_plot = gr.Textbox(placeholder="The plot of the third chapter will be generated here", lines=3, elem_classes=["no-label"])
+
+		gr.Textbox("Falling action", elem_classes=["no-label"])
+		with gr.Row(elem_classes=["left-margin"]):  
+			chapter4_plot = gr.Textbox(placeholder="The plot of the fourth chapter will be generated here", lines=3, elem_classes=["no-label"])
+
+		gr.Textbox("Denouement", elem_classes=["no-label"])
+		with gr.Row(elem_classes=["left-margin"]):  
+			chapter5_plot = gr.Textbox(placeholder="The plot of the fifth chapter will be generated here", lines=3, elem_classes=["no-label"])
+
+		with gr.Row():
+			plot_gen_temp = gr.Slider(0.0, 2.0, 1.0, step=0.1, label="temperature")
+			plot_gen_btn = gr.Button("gen plot", elem_classes=["control-button"])
+
+		plot_setup_confirm_btn = gr.Button("confirm", elem_classes=["control-button"])
+
+	with gr.Column(visible=False) as writing_phase:
+		gr.Markdown("# ✍🏼 Story writing")
+		gr.Markdown(desc.story_generation_phase_description, elem_classes=["markdown-justify"])
+  
+		progress_comp = gr.Textbox(label=None, elem_classes=["no-label"], interactive=False)
+
+		title_display = gr.Markdown("# Title Undetermined Yet", elem_classes=["markdown-center"], visible=False)
+		subtitle_display = gr.Markdown("### Title Undetermined Yet", elem_classes=["markdown-center"], visible=False)
+
+		with gr.Row():
+			image_gen_btn = gr.Button("🏞️ Image", interactive=False, elem_classes=["control-button-green"])
+			audio_gen_btn = gr.Button("🔊 Audio", interactive=False, elem_classes=["control-button-green"])
+			img_audio_combine_btn = gr.Button("📀 Image + Audio", interactive=False, elem_classes=["control-button-green"])
+
+		story_image = gr.Image(None, visible=False, type="filepath", interactive=False, elem_classes=["no-label-image-audio"])
+		story_audio = gr.Audio(None, visible=False, type="filepath", interactive=False, elem_classes=["no-label-image-audio"])
+		story_video = gr.Video(visible=False, interactive=False, elem_classes=["no-label-gallery"])
+
+		story_progress = gr.Slider(
+			1, 2, 1, step=1, interactive=True, 
+			label="1/2", visible=False
+		)
+
+		story_content = gr.Textbox(
+				"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer interdum eleifend tincidunt. Vivamus dapibus, massa ut imperdiet condimentum, quam ipsum vehicula eros, a accumsan nisl metus at nisl. Nullam tortor nibh, vehicula sed tellus at, accumsan efficitur enim. Sed mollis purus vitae nisl ornare volutpat. In vitae tortor nec neque sagittis vehicula. In vestibulum velit eu lorem pulvinar dignissim. Donec eu sapien et sapien cursus pretium elementum eu urna. Proin lacinia ipsum maximus, commodo dui tempus, convallis tortor. Nulla sodales mi libero, nec eleifend eros interdum quis. Pellentesque nulla lectus, scelerisque et consequat vitae, blandit at ante. Sed nec …….",
+				lines=12,
+				elem_classes=["no-label", "small-big-textarea"]
+		)		
+
+		action_types = gr.Radio(
+			choices=[
+				"continue current phase", "move to the next phase"
+			],
+			value="continue current phase",
+			interactive=True,
+			elem_classes=["no-label-radio"],
+			visible=False,
+		)
+
+		with gr.Accordion("regeneration controls", open=False):
+			with gr.Row():
+				regen_actions_btn = gr.Button("Re-suggest actions", interactive=True, elem_classes=["control-button-green"])
+				regen_story_btn = gr.Button("Re-suggest story and actions", interactive=True, elem_classes=["control-button-green"])
+	
+			custom_prompt_txt = gr.Textbox(placeholder="Re-suggest story and actions based on your own custom request", elem_classes=["no-label", "small-big-textarea"])
+
+		with gr.Row():
+			action_btn1 = gr.Button("Action Choice 1", interactive=False, elem_classes=["control-button-green"])
+			action_btn2 = gr.Button("Action Choice 2", interactive=False, elem_classes=["control-button-green"])
+			action_btn3 = gr.Button("Action Choice 3", interactive=False, elem_classes=["control-button-green"])
+
+		custom_action_txt = gr.Textbox(placeholder="write your own custom action", elem_classes=["no-label", "small-big-textarea"], scale=3)
+
+		with gr.Row():
+			restart_from_story_generation_btn = gr.Button("← back", elem_classes=["wrap", "control-button"], scale=1)
+			story_writing_done_btn = gr.Button("export your story →", elem_classes=["wrap", "control-button"], scale=2)
+
+	with gr.Column(visible=False) as export_phase:
+		gr.Markdown("### 📤 Export output")
+		with gr.Accordion("generate chapter titles and each plot", open=False) as export_section:
+			gr.Markdown("hello")
+
+	with gr.Accordion("💬", open=False, elem_id="chat-section") as chat_section:
+		with gr.Column(scale=1):
+			chatbot = gr.Chatbot(
+				[],
+				avatar_images=("assets/user.png", "assets/ai.png"), 
+				elem_id="chatbot", 
+				elem_classes=["no-label-chatbot"])
+			chat_input_txt = gr.Textbox(placeholder="enter...", interactive=True, elem_id="chat-input", elem_classes=["no-label"])
+
+			with gr.Row(elem_id="chat-buttons"):
+				regen_btn = gr.Button("regen", interactive=False, elem_classes=["control-button"])
+				clear_btn = gr.Button("clear", elem_classes=["control-button"])
+
+	pre_to_setup_btn.click(
+		view_change_ui.move_to_next_view,
+		inputs=None,
+		outputs=[pre_phase, background_setup_phase]
+	)
+ 
+	back_to_pre_btn.click(
+		view_change_ui.back_to_previous_view,
+		inputs=None,
+		outputs=[pre_phase, background_setup_phase]
+	)
+
+	world_setup_confirm_btn.click(
+		view_change_ui.move_to_next_view,
+		inputs=None,
+		outputs=[background_setup_phase, character_setup_phase]
+	)
+ 
+	back_to_background_setup_btn.click(
+		view_change_ui.back_to_previous_view,
+		inputs=None,
+		outputs=[background_setup_phase, character_setup_phase]		
+	)
+ 
+	restart_from_story_generation_btn.click(
+		view_change_ui.move_to_next_view,
+		inputs=None,
+		outputs=[pre_phase, writing_phase]		
+	)
+
+	character_setup_confirm_btn.click(
+		view_change_ui.move_to_next_view,
+		inputs=None,
+		outputs=[character_setup_phase, writing_phase]
+	).then(
+		story_gen_ui.first_story_gen,
+		inputs=[			
+			cursors,
+			genre_dd, place_dd, mood_dd, 
+			name_txt1, age_dd1, mbti_dd1, personality_dd1, job_dd1,
+			side_char_enable_ckb1, name_txt2, age_dd2, mbti_dd2, personality_dd2, job_dd2,
+			side_char_enable_ckb2, name_txt3, age_dd3, mbti_dd3, personality_dd3, job_dd3,
+			side_char_enable_ckb3, name_txt4, age_dd4, mbti_dd4, personality_dd4, job_dd4,
+		],
+		outputs=[
+			cursors, cur_cursor, story_content, story_progress, image_gen_btn, audio_gen_btn,
+			story_image, story_audio, story_video
+		]
+	).then(
+		story_gen_ui.actions_gen,
+		inputs=[
+			cursors,
+			genre_dd, place_dd, mood_dd, 
+			name_txt1, age_dd1, mbti_dd1, personality_dd1, job_dd1,
+			side_char_enable_ckb1, name_txt2, age_dd2, mbti_dd2, personality_dd2, job_dd2,
+			side_char_enable_ckb2, name_txt3, age_dd3, mbti_dd3, personality_dd3, job_dd3,
+			side_char_enable_ckb3, name_txt4, age_dd4, mbti_dd4, personality_dd4, job_dd4,
+		],
+		outputs=[
+			action_btn1, action_btn2, action_btn3, progress_comp
+		]
+	)
+
+	regen_actions_btn.click(
+		story_gen_ui.actions_gen,
+		inputs=[
+			cursors,
+			genre_dd, place_dd, mood_dd, 
+			name_txt1, age_dd1, mbti_dd1, personality_dd1, job_dd1,
+			side_char_enable_ckb1, name_txt2, age_dd2, mbti_dd2, personality_dd2, job_dd2,
+			side_char_enable_ckb2, name_txt3, age_dd3, mbti_dd3, personality_dd3, job_dd3,
+			side_char_enable_ckb3, name_txt4, age_dd4, mbti_dd4, personality_dd4, job_dd4,
+		],
+		outputs=[
+			action_btn1, action_btn2, action_btn3, progress_comp
+		]		
+	)
+ 
+	regen_story_btn.click(
+		story_gen_ui.update_story_gen,
+		inputs=[
+			cursors, cur_cursor,
+			genre_dd, place_dd, mood_dd, 
+			name_txt1, age_dd1, mbti_dd1, personality_dd1, job_dd1,
+			side_char_enable_ckb1, name_txt2, age_dd2, mbti_dd2, personality_dd2, job_dd2,
+			side_char_enable_ckb2, name_txt3, age_dd3, mbti_dd3, personality_dd3, job_dd3,
+			side_char_enable_ckb3, name_txt4, age_dd4, mbti_dd4, personality_dd4, job_dd4,			
+		],
+		outputs=[
+			cursors, cur_cursor, story_content, story_progress, image_gen_btn, audio_gen_btn
+		]
+	).then(
+		story_gen_ui.actions_gen,
+		inputs=[
+			cursors,
+			genre_dd, place_dd, mood_dd, 
+			name_txt1, age_dd1, mbti_dd1, personality_dd1, job_dd1,
+			side_char_enable_ckb1, name_txt2, age_dd2, mbti_dd2, personality_dd2, job_dd2,
+			side_char_enable_ckb2, name_txt3, age_dd3, mbti_dd3, personality_dd3, job_dd3,
+			side_char_enable_ckb3, name_txt4, age_dd4, mbti_dd4, personality_dd4, job_dd4,
+		],
+		outputs=[
+			action_btn1, action_btn2, action_btn3, progress_comp
+		]
+	)
+ 
+	#### Setups
+
+	genre_dd.select(
+		ui.update_on_age,
+		outputs=[place_dd, mood_dd, job_dd1, job_dd2, job_dd3, job_dd4]
+	)
+
+	gen_char_btn1.click(
+		ui.gen_character_image,
+		inputs=[
+			gallery_images1, name_txt1, age_dd1, mbti_dd1, personality_dd1, job_dd1, genre_dd, place_dd, mood_dd, creative_dd1],
+		outputs=[char_gallery1, gallery_images1]
+	)
+
+	gen_char_btn2.click(
+		ui.gen_character_image,
+		inputs=[gallery_images2, name_txt2, age_dd2, mbti_dd2, personality_dd2, job_dd2, genre_dd, place_dd, mood_dd, creative_dd2],
+		outputs=[char_gallery2, gallery_images2]
+	)
+
+	gen_char_btn3.click(
+		ui.gen_character_image,
+		inputs=[gallery_images3, name_txt3, age_dd3, mbti_dd3, personality_dd3, job_dd3, genre_dd, place_dd, mood_dd, creative_dd3],
+		outputs=[char_gallery3, gallery_images3]
+	)
+
+	gen_char_btn4.click(
+		ui.gen_character_image,
+		inputs=[gallery_images4, name_txt4, age_dd4, mbti_dd4, personality_dd4, job_dd4, genre_dd, place_dd, mood_dd, creative_dd4],
+		outputs=[char_gallery4, gallery_images4]
+	)
+
+	random_name_btn1.click(
+		ui.get_random_name,
+		inputs=[name_txt1, name_txt2, name_txt3, name_txt4],
+		outputs=[name_txt1],
+	)
+
+	random_name_btn2.click(
+		ui.get_random_name,
+		inputs=[name_txt2, name_txt1, name_txt3, name_txt4],
+		outputs=[name_txt2],
+	)
+
+	random_name_btn3.click(
+		ui.get_random_name,
+		inputs=[name_txt3, name_txt1, name_txt2, name_txt4],
+		outputs=[name_txt3],
+	)
+
+	random_name_btn4.click(
+		ui.get_random_name,
+		inputs=[name_txt4, name_txt1, name_txt2, name_txt3],
+		outputs=[name_txt4],
+	)
+ 
+	### Story generation
+	story_content.input(
+		story_gen_ui.update_story_content,
+		inputs=[story_content, cursors, cur_cursor],
+		outputs=[cursors],
+	)
+
+	image_gen_btn.click(
+		story_gen_ui.image_gen,
+		inputs=[
+			genre_dd, place_dd, mood_dd, title, story_content, cursors, cur_cursor, story_audio
+		],
+		outputs=[
+			story_image, img_audio_combine_btn, cursors, progress_comp,
+		]
+	)
+
+	audio_gen_btn.click(
+		story_gen_ui.audio_gen,
+		inputs=[
+			genre_dd, place_dd, mood_dd, title, story_content, cursors, cur_cursor, story_image
+		],
+		outputs=[story_audio, img_audio_combine_btn, cursors, progress_comp]
+	)
+
+	img_audio_combine_btn.click(
+		story_gen_ui.video_gen,
+		inputs=[
+			story_image, story_audio, story_content, cursors, cur_cursor
+		],
+		outputs=[
+			story_image, story_audio, story_video, cursors, progress_comp
+		],
+	)	
+
+	story_progress.input(
+		story_gen_ui.move_story_cursor,
+		inputs=[
+			story_progress, cursors
+		],
+		outputs=[
+			cur_cursor,
+			story_progress, 
+			story_content,
+			story_image, story_audio, story_video,
+			action_btn1, action_btn2, action_btn3,
+		]
+	)
+
+	action_btn1.click(
+		lambda: (gr.update(interactive=False), gr.update(interactive=False), gr.update(interactive=False)),
+		inputs=None,
+		outputs=[
+			image_gen_btn, audio_gen_btn, img_audio_combine_btn
+		]
+	).then(
+		story_gen_ui.next_story_gen,
+		inputs=[
+			cursors,
+			action_btn1,
+			genre_dd, place_dd, mood_dd, 
+			name_txt1, age_dd1, mbti_dd1, personality_dd1, job_dd1,
+			side_char_enable_ckb1, name_txt2, age_dd2, mbti_dd2, personality_dd2, job_dd2,
+			side_char_enable_ckb2, name_txt3, age_dd3, mbti_dd3, personality_dd3, job_dd3,
+			side_char_enable_ckb3, name_txt4, age_dd4, mbti_dd4, personality_dd4, job_dd4,	
+		],
+		outputs=[
+			cursors, cur_cursor,
+			story_content, story_progress,
+			image_gen_btn, audio_gen_btn,
+			story_image, story_audio, story_video
+		]
+	).then(
+		story_gen_ui.actions_gen,
+		inputs=[
+			cursors,
+			genre_dd, place_dd, mood_dd, 
+			name_txt1, age_dd1, mbti_dd1, personality_dd1, job_dd1,
+			side_char_enable_ckb1, name_txt2, age_dd2, mbti_dd2, personality_dd2, job_dd2,
+			side_char_enable_ckb2, name_txt3, age_dd3, mbti_dd3, personality_dd3, job_dd3,
+			side_char_enable_ckb3, name_txt4, age_dd4, mbti_dd4, personality_dd4, job_dd4,
+		],
+		outputs=[
+			action_btn1, action_btn2, action_btn3, progress_comp
+		]
+	)
+
+	action_btn2.click(
+		lambda: (gr.update(interactive=False), gr.update(interactive=False), gr.update(interactive=False)),
+		inputs=None,
+		outputs=[
+			image_gen_btn, audio_gen_btn, img_audio_combine_btn
+		]
+	).then(
+		story_gen_ui.next_story_gen,
+		inputs=[
+			cursors,
+			action_btn2,
+			genre_dd, place_dd, mood_dd, 
+			name_txt1, age_dd1, mbti_dd1, personality_dd1, job_dd1,
+			side_char_enable_ckb1, name_txt2, age_dd2, mbti_dd2, personality_dd2, job_dd2,
+			side_char_enable_ckb2, name_txt3, age_dd3, mbti_dd3, personality_dd3, job_dd3,
+			side_char_enable_ckb3, name_txt4, age_dd4, mbti_dd4, personality_dd4, job_dd4,	
+		],
+		outputs=[
+			cursors, cur_cursor,
+			story_content, story_progress,
+			image_gen_btn, audio_gen_btn,
+			story_image, story_audio, story_video
+		]
+	).then(
+		story_gen_ui.actions_gen,
+		inputs=[
+			cursors,
+			genre_dd, place_dd, mood_dd, 
+			name_txt1, age_dd1, mbti_dd1, personality_dd1, job_dd1,
+			side_char_enable_ckb1, name_txt2, age_dd2, mbti_dd2, personality_dd2, job_dd2,
+			side_char_enable_ckb2, name_txt3, age_dd3, mbti_dd3, personality_dd3, job_dd3,
+			side_char_enable_ckb3, name_txt4, age_dd4, mbti_dd4, personality_dd4, job_dd4,
+		],
+		outputs=[
+			action_btn1, action_btn2, action_btn3, progress_comp
+		]
+	)
+
+	action_btn3.click(
+		lambda: (gr.update(interactive=False), gr.update(interactive=False), gr.update(interactive=False)),
+		inputs=None,
+		outputs=[
+			image_gen_btn, audio_gen_btn, img_audio_combine_btn
+		]
+	).then(
+		story_gen_ui.next_story_gen,
+		inputs=[
+			cursors,
+			action_btn3,
+			genre_dd, place_dd, mood_dd, 
+			name_txt1, age_dd1, mbti_dd1, personality_dd1, job_dd1,
+			side_char_enable_ckb1, name_txt2, age_dd2, mbti_dd2, personality_dd2, job_dd2,
+			side_char_enable_ckb2, name_txt3, age_dd3, mbti_dd3, personality_dd3, job_dd3,
+			side_char_enable_ckb3, name_txt4, age_dd4, mbti_dd4, personality_dd4, job_dd4,	
+		],
+		outputs=[
+			cursors, cur_cursor,
+			story_content, story_progress,
+			image_gen_btn, audio_gen_btn,
+			story_image, story_audio, story_video
+		]
+	).then(
+		story_gen_ui.actions_gen,
+		inputs=[
+			cursors,
+			genre_dd, place_dd, mood_dd, 
+			name_txt1, age_dd1, mbti_dd1, personality_dd1, job_dd1,
+			side_char_enable_ckb1, name_txt2, age_dd2, mbti_dd2, personality_dd2, job_dd2,
+			side_char_enable_ckb2, name_txt3, age_dd3, mbti_dd3, personality_dd3, job_dd3,
+			side_char_enable_ckb3, name_txt4, age_dd4, mbti_dd4, personality_dd4, job_dd4,
+		],
+		outputs=[
+			action_btn1, action_btn2, action_btn3, progress_comp
+		]
+	)
+
+	custom_action_txt.submit(
+		lambda: (gr.update(interactive=False), gr.update(interactive=False), gr.update(interactive=False)),
+		inputs=None,
+		outputs=[
+			image_gen_btn, audio_gen_btn, img_audio_combine_btn
+		]
+	).then(
+		story_gen_ui.next_story_gen,
+		inputs=[
+			cursors,
+			custom_action_txt,
+			genre_dd, place_dd, mood_dd, 
+			name_txt1, age_dd1, mbti_dd1, personality_dd1, job_dd1,
+			side_char_enable_ckb1, name_txt2, age_dd2, mbti_dd2, personality_dd2, job_dd2,
+			side_char_enable_ckb2, name_txt3, age_dd3, mbti_dd3, personality_dd3, job_dd3,
+			side_char_enable_ckb3, name_txt4, age_dd4, mbti_dd4, personality_dd4, job_dd4,	
+		],
+		outputs=[
+			cursors, cur_cursor,
+			story_content, story_progress,
+			image_gen_btn, audio_gen_btn,
+			story_image, story_audio, story_video
+		]
+	).then(
+		story_gen_ui.actions_gen,
+		inputs=[
+			cursors,
+			genre_dd, place_dd, mood_dd, 
+			name_txt1, age_dd1, mbti_dd1, personality_dd1, job_dd1,
+			side_char_enable_ckb1, name_txt2, age_dd2, mbti_dd2, personality_dd2, job_dd2,
+			side_char_enable_ckb2, name_txt3, age_dd3, mbti_dd3, personality_dd3, job_dd3,
+			side_char_enable_ckb3, name_txt4, age_dd4, mbti_dd4, personality_dd4, job_dd4,
+		],
+		outputs=[
+			action_btn1, action_btn2, action_btn3, progress_comp
+		]
+	)
+
+	### Chatbot
+
+	# chat_input_txt.submit(
+	# 	chat_ui.chat,
+	# 	inputs=[
+	# 		chat_input_txt, chat_mode, chat_state,
+	# 		genre_dd, place_dd, mood_dd, 
+	# 		name_txt1, age_dd1, mbti_dd1, personality_dd1, job_dd1,
+	# 		name_txt2, age_dd2, mbti_dd2, personality_dd2, job_dd2,
+	# 		name_txt3, age_dd3, mbti_dd3, personality_dd3, job_dd3,
+	# 		name_txt4, age_dd4, mbti_dd4, personality_dd4, job_dd4,
+	# 		chapter1_title, chapter2_title, chapter3_title, chapter4_title,
+	# 		chapter1_plot, chapter2_plot, chapter3_plot, chapter4_plot
+	# 	],
+	# 	outputs=[chat_input_txt, chat_state, chatbot, regen_btn]
+	# )
+ 
+	regen_btn.click(
+		chat_ui.rollback_last_ui,
+		inputs=[chatbot], outputs=[chatbot]
+	).then(
+		chat_ui.chat_regen,
+		inputs=[chat_mode, chat_state],
+		outputs=[chat_state, chatbot]		
+	)
+ 
+	clear_btn.click(
+		chat_ui.chat_reset,
+		inputs=[chat_mode, chat_state],
+		outputs=[chat_input_txt, chat_state, chatbot, regen_btn]
+	)
 
-iface = gr.Interface(fn=greet, inputs="text", outputs="text")
-iface.launch()
+demo.queue().launch(share=True)

assets/.gitattributes

@@ -0,0 +1,7 @@
+image.png filter=lfs diff=lfs merge=lfs -text
+nsfw_warning.png filter=lfs diff=lfs merge=lfs -text
+nsfw_warning_wide.png filter=lfs diff=lfs merge=lfs -text
+overview.png filter=lfs diff=lfs merge=lfs -text
+user.png filter=lfs diff=lfs merge=lfs -text
+ai.png filter=lfs diff=lfs merge=lfs -text
+background.png filter=lfs diff=lfs merge=lfs -text

assets/palm_prompts.toml

@@ -0,0 +1,154 @@
+[image_gen]
+neg_prompt="nsfw, worst quality, low quality, lowres, bad anatomy, bad hands, text, watermark, signature, error, missing fingers, extra digit, fewer digits, cropped, worst quality, normal quality, blurry, username, extra limbs, twins, boring, jpeg artifacts"
+
+[image_gen.character]
+gen_prompt = """Based on my brief descriptions of the character, suggest a "primary descriptive sentence" and "concise descriptors" to visualize them. Ensure you consider elements like the character's gender, age, appearance, occupation, clothing, posture, facial expression, mood, among others.
+Once complete, please output only a single "primary descriptive sentence" and the "concise descriptors" in a syntactically valid JSON format.
+The output template is as follows: {{"primary_sentence":"primary descriptive sentence","descriptors":["concise descriptor 1","concise descriptor 2","concise descriptor 3"]}}.
+To enhance the quality of your character's description or expression, you might consider drawing from the following categories:
+- Emotions and Expressions: "ecstatic", "melancholic", "furious", "startled", "bewildered", "pensive", "overjoyed", "crushed", "elated", "panicked", "satisfied", "cynical", "apathetic", "delighted", "terrified", "desperate", "triumphant", "mortified", "envious", "appreciative", "blissful", "heartbroken", "livid", "astounded", "baffled", "smiling", "frowning", "grinning", "crying", "pouting", "glaring", "blinking", "winking", "smirking", "whistling".
+- Physical Features: "upper body", "very long hair", "looking at viewer", "looking to the side", "looking at another", "thick lips", "skin spots", "acnes", "skin blemishes", "age spot", "perfect eyes", "detailed eyes", "realistic eyes", "dynamic standing", "beautiful face", "necklace", "high detailed skin", "hair ornament", "blush", "shiny skin", "long sleeves", "cleavage", "rubber suit", "slim", "plump", "muscular", "pale skin", "tan skin", "dark skin", "blonde hair", "brunette hair", "black hair", "blue eyes", "green eyes", "brown eyes", "curly hair", "short hair", "wavy hair".
+- Visual Enhancements: "masterpiece", "cinematic lighting", "detailed lighting", "tyndall effect", "soft lighting", "volumetric lighting", "close up", "wide shot", "glossy", "beautiful lighting", "warm lighting", "extreme", "ultimate", "best", "supreme", "ultra", "intense", "powerful", "exceptional", "remarkable", "strong", "vigorous", "dynamic angle", "front view person", "bangs", "waist up", "bokeh".
+- Age and Gender: "1boy", "1man", "1male", "1girl", "1woman", "1female", "teen", "teenage", "twenties", "thirties", "forties", "fifties", "middle-age".
+Do note that this list isn't exhaustive, and you're encouraged to suggest similar terms not included here.
+Exclude words from the suggestion that are redundant or have conflicting meanings.
+Especially, Exclude words that conflict with the meaning of "main_sentence".
+Do not output anything other than JSON values.
+Do not provide any additional explanation of the following.
+Only JSON is allowed.
+===
+This is some examples.
+Q:
+The character's name is Liam, their job is as the Secret Agent, and they are in their 50s. And the keywords that help in associating with the character are "Thriller, Underground Warehouse, Darkness, ESTP, Ambitious, Generous".
+Print out no more than 45 words in syntactically valid JSON format.
+A:
+{{"primary_sentence":"Middle-aged man pointing a gun in an underground warehouse","descriptors":["1man","solo","masterpiece","best quality","upper body","black suit","pistol in hand","dramatic lighting","muscular physique","intense brown eyes","raven-black hair","stylish cut","determined gaze","looking at viewer","stealthy demeanor","cunning strategist","advanced techwear","sleek","night operative","shadowy figure","night atmosphere","mysterious aura","highly detailed","film grain","detailed eyes and face"]}}
+
+Q:
+The character's name is Catherine, their job is as the Traveler, and they are in their 10s. And the keywords that help in associating with the character are "Romance, Starlit Bridge, Dreamy, ENTJ, Ambitious".
+Print out no more than 45 words in syntactically valid JSON format.
+A:
+{{"primary_sentence":"A dreamy teenage girl standing on a starlit bridge with romantic ambitions","descriptors":["1girl","solo","masterpiece","best quality","upper body","flowing skirt","sun hat","bright-eyed","map in hand","ethereal beauty","wanderlust","scarf","whimsical","graceful poise","celestial allure","close-up","warm soft lighting","luminescent glow","gentle aura","mystic charm","smirk","dreamy landscape","poetic demeanor","cinematic lighting","extremely detailed","film grain","detailed eyes and face"]}}
+
+Q:
+The character's name is Claire, their job is as the Technological Advancement, and they are in their 20s. And the keywords that help in associating with the character are "Science Fiction, Space Station, INFP, Ambitious, Generous".
+Print out no more than 45 words in syntactically valid JSON format.
+A:
+{{"primary_sentence":"A young ambitious woman tech expert aboard a futuristic space station","descriptors":["1girl","solo","masterpiece","best quality","upper body","sleek silver jumpsuit","futuristic heels","contemplative","editorial portrait","dynamic angle","sci-fi","techno-savvy","sharp focus","bokeh","beautiful lighting","intricate circuitry","robotic grace","rich colors","vivid contrasts","dramatic lighting","futuristic flair","avant-garde","high-tech allure","innovative mind","mechanical sophistication","film grain","detailed eyes and face"]}}
+
+Q:
+The character's name is Sophie, their job is as a Ballet Dancer, and they are in their 10s. And the keywords that help in associating with the character are "Grace, Dance Studio, Elegance, ISFJ, Gentle, Passionate"
+Print out no more than 45 words in syntactically valid JSON format.
+A:
+{{"primary_sentence":"An elegant dancer poses gracefully in a mirrored studio","descriptors":["1girl","teen","solo","masterpiece","best quality","upper body","beautiful face","shiny skin","wavy hair","ballet attire","tiptoe stance","flowing skirt","focused gaze","soft ambiance","soft lighting","film grain","detailed eyes and face"]}}
+===
+This is my request.
+Q:
+{input}
+A:
+"""
+query = """
+The character's name is {character_name}, their job is as the {job}, and they are in their {age}. And the keywords that help in associating with the character are "{keywords}".
+Print out no more than 45 words in syntactically valid JSON format.
+"""
+
+[image_gen.background]
+gen_prompt = """Based on my brief descriptions of the scene, suggest a "primary descriptive sentence" and "concise descriptors" to visualize it. Ensure you consider elements like the setting's time of day, atmosphere, prominent objects, mood, location, natural phenomena, architecture, among others.
+Once complete, please output only a single "primary descriptive sentence" and the "concise descriptors" in a syntactically valid JSON format.
+The output template is as follows: {{"primary_sentence":"primary descriptive sentence","descriptors":["concise descriptor 1","concise descriptor 2","concise descriptor 3"]}}.
+To enhance the quality of your scene's description or expression, you might consider drawing from the following categories:
+- Atmosphere and Time: "dawn", "dusk", "midday", "midnight", "sunset", "sunrise", "foggy", "misty", "stormy", "calm", "clear night", "starlit", "moonlit", "golden hour".
+- Natural Phenomena: "rainbow", "thunderstorm", "snowfall", "aurora borealis", "shooting star", "rain shower", "windy", "sunny".
+- Location and Architecture: "urban", "rural", "mountainous", "oceanfront", "forest", "desert", "island", "modern city", "ancient ruins", "castle", "village", "meadow", "cave", "bridge".
+- Prominent Objects: "giant tree", "waterfall", "stream", "rock formation", "ancient artifact", "bonfire", "tent", "vehicle", "statue", "fountain".
+- Visual Enhancements: "masterpiece", "cinematic lighting", "detailed lighting", "soft lighting", "volumetric lighting", "tyndall effect", "warm lighting", "close up", "wide shot", "beautiful perspective", "bokeh".
+Do note that this list isn't exhaustive, and you're encouraged to suggest similar terms not included here.
+Exclude words from the suggestion that are redundant or have conflicting meanings.
+Especially, Exclude words that conflict with the meaning of "main_sentence".
+Do not output anything other than JSON values.
+Do not provide any additional explanation of the following.
+Only JSON is allowed.
+===
+This is some examples.
+Q:
+The genre is "Fantasy", the place is "Enchanted Forest", the mood is "Mystical", the title of the novel is "Whispering Leaves", and the chapter plot revolves around "A hidden glade where elves sing under the moonlight".
+Print out no more than 45 words in syntactically valid JSON format.
+A:
+{{"main_sentence":"a mystical glade in an enchanted forest where elves sing beneath the moonlight","descriptors":["no humans","masterpiece","fantasy","enchanted forest","moonlit glade","mystical atmosphere","singing elves","luminous fireflies","ancient trees","shimmering leaves","whispering winds","hidden secrets","elven magic","masterpiece","soft lighting","silver glow","detailed shadows","enchanted mood","highly detailed","film grain"]}}
+
+Q:
+The genre is "Science Fiction", the place is "Galactic Space Station", the mood is "Tense", the title of the novel is "Stars Unbound", and the chapter plot revolves around "Ambassadors from different galaxies discussing a new treaty".
+Print out no more than 45 words in syntactically valid JSON format.
+A:
+{{"main_sentence":"a tense gathering in a galactic space station where interstellar ambassadors negotiate","descriptors":["no humans","masterpiece","science fiction","galactic space station","star-studded backdrop","advanced technology","diverse aliens","hovering spacecrafts","futuristic architecture","tense discussions","interstellar politics","neon lights","holographic displays","masterpiece","detailed lighting","cinematic mood","highly detailed","film grain"]}}
+
+Q:
+The genre is "Romance", the place is "Beach", the mood is "Heartfelt", the title of the novel is "Waves of Passion", and the chapter plot revolves around "Two lovers reconciling their differences by the shore".
+Print out no more than 45 words in syntactically valid JSON format.
+A:
+{{"main_sentence":"a heartfelt scene on a beach during sunset where two lovers reconcile","descriptors":["no humans","masterpiece","romance","beach","sunset horizon","golden sands","lapping waves","embrace","teary-eyed confessions","seashells","reflective waters","warm hues","silhouette of lovers","soft breeze","beautiful perspective","detailed shadows","emotional atmosphere","highly detailed","film grain"]}}
+
+Q:
+The genre is "Middle Ages", the place is "Royal Palace", the mood is "Epic Adventure", the title of the novel is "Throne of Fates", and the chapter plot revolves around "A brave knight receiving a quest from the king".
+Print out no more than 45 words in syntactically valid JSON format.
+A:
+{{"main_sentence":"an epic scene in a royal palace where a knight is tasked with a quest by the king","descriptors":["no humans","masterpiece","middle ages","royal palace","castle","grand throne room","golden hour","armored knight","majestic king","tapestries","stone walls","torches","glistening armor","banner flags","medieval atmosphere","heroic demeanor","detailed architecture","golden crowns","highly detailed","film grain"]}}
+===
+This is my request.
+Q:
+{input}
+A:
+"""
+query = """
+The genre is "{genre}", the place is "{place}", the mood is "{mood}", the title of the novel is "{title}", and the chapter plot revolves around "{chapter_plot}".
+Print out no more than 45 words in syntactically valid JSON format.
+"""
+
+[music_gen]
+gen_prompt = """Based on my brief descriptions of the novel's mood, theme, or setting, suggest a "primary descriptive sentence" to conceptualize the musical piece. Ensure you consider elements like the music's genre, BPM, primary instruments, emotions evoked, era (if applicable), and other relevant musical characteristics.
+Once complete, please output only a single "primary descriptive sentence" in a syntactically valid JSON format.
+The output template is as follows:
+{{"primary_sentence":"primary descriptive sentence"}}.
+To enhance the quality of your music's description or expression, you might consider drawing from the following categories:
+- Musical Genre and Era: "80s", "90s", "classical", "jazz", "EDM", "rock", "folk", "baroque", "bebop", "grunge", "funk", "hip-hop", "blues", "country".
+- BPM and Rhythm: "slow-paced", "mid-tempo", "upbeat", "rhythmic", "syncopated", "steady beat", "dynamic tempo".
+- Primary Instruments and Sound: "guitar", "synth", "piano", "saxophone", "drums", "violin", "flute", "bassy", "treble-heavy", "distorted", "acoustic", "electric", "ambient sounds".
+- Emotions and Atmosphere: "nostalgic", "energetic", "melancholic", "uplifting", "dark", "light-hearted", "intense", "relaxing", "haunting", "joyful", "sombre", "celebratory", "mystical".
+- Musical Techniques and Enhancements: "harmonious", "dissonant", "layered", "minimalistic", "rich textures", "simple melody", "complex rhythms", "vocal harmonies", "instrumental solo".
+Do note that this list isn't exhaustive, and you're encouraged to suggest similar terms not included here.
+Exclude words from the suggestion that are redundant or have conflicting meanings.
+Especially, Exclude words that conflict with the meaning of "primary_sentence".
+Do not output anything other than JSON values.
+Do not provide any additional explanation of the following.
+Only JSON is allowed.
+===
+This is some examples.
+Q:
+The genre is "Fantasy", the place is "Enchanted Forest", the mood is "Mystical", the title of the novel is "Whispering Leaves", and the chapter plot revolves around "A hidden glade where elves sing under the moonlight".
+A:
+{{"main_sentence":"a gentle folk melody filled with whimsical flutes, echoing harps, and distant ethereal vocals, capturing the enchantment of a moonlit forest and the mystique of singing elves"}}
+
+Q:
+The genre is "Science Fiction", the place is "Galactic Space Station", the mood is "Tense", the title of the novel is "Stars Unbound", and the chapter plot revolves around "Ambassadors from different galaxies discussing a new treaty".
+A:
+{{"main_sentence":"an ambient electronic track, with pulsating synths, spacey reverberations, and occasional digital glitches, reflecting the vastness of space and the tension of intergalactic diplomacy"}}
+
+Q:
+The genre is "Romance", the place is "Beach", the mood is "Heartfelt", the title of the novel is "Waves of Passion", and the chapter plot revolves around "Two lovers reconciling their differences by the shore".
+A:
+{{"main_sentence":"a soft acoustic ballad featuring soulful guitars, delicate percussion, and heartfelt vocals, evoking feelings of love, reconciliation, and the gentle ebb and flow of the ocean waves"}}
+
+Q:
+The genre is "Middle Ages", the place is "Royal Palace", the mood is "Epic Adventure", the title of the novel is "Throne of Fates", and the chapter plot revolves around "A brave knight receiving a quest from the king".
+A:
+{{"main_sentence":"a grand orchestral piece, dominated by powerful brass, rhythmic drums, and soaring strings, portraying the valor of knights, the majesty of royalty, and the anticipation of an epic quest"}}
+===
+This is my request.
+Q:
+{input}
+A:
+"""
+query = """
+The genre is "{genre}", the place is "{place}", the mood is "{mood}", the title of the novel is "{title}", and the chapter plot revolves around "{chapter_plot}".
+Print out only one main_sentence in syntactically valid JSON format.
+"""

constants/css.py

@@ -0,0 +1,186 @@
+STYLE = """
+.main {
+  width: 75% !important;
+  margin: auto;
+}
+
+.ninty-five-width {
+  width: 95% !important;
+  margin: auto;
+}
+
+.center-label > label > span {
+  display: block !important;
+  text-align: center;
+}
+
+.no-label {
+  padding: 0px !important;
+}
+
+.no-label > label > span {
+  display: none;
+}
+
+.wrap {
+  min-width: 0px !important;
+}
+
+.markdown-center {
+  text-align: center;
+}
+
+.markdown-justify {
+  text-align: justify !important;
+}
+
+.markdown-left {
+  text-align: left;
+}
+
+.markdown-left > div:nth-child(2) {
+  padding-top: 10px !important;
+}
+
+.markdown-center > div:nth-child(2) {
+  padding-top: 10px;
+}
+
+.no-gap {
+  flex-wrap: initial !important;
+  gap: initial !important;
+}
+
+.no-width {
+  min-width: 0px !important;
+}
+
+.icon-buttons {
+  display: none !important;
+}
+
+.title-width {
+  display: content !important;
+}
+
+.left-margin {
+  padding-left: 50px;
+  background-color: transparent;
+  border: none;
+}
+
+.no-border > div:nth-child(1){
+  border: none;
+  background: transparent;
+}
+
+textarea {
+  border: none !important;
+  border-radius: 0px !important;
+  --block-background-fill: transparent !important;
+}
+
+#chatbot {
+  height: 800px !important;
+  box-shadow: 6px 5px 10px 1px rgba(255, 221, 71, 0.15);
+  border-color: beige;
+  border-width: 2px;  
+}
+
+#chatbot .wrapper {
+  height: 660px;
+}
+
+.small-big-textarea > label > textarea {
+  font-size: 12pt !important;
+}
+
+.control-button {
+  background: none !important;
+  border-color: #69ade2 !important;
+  border-width: 2px !important;
+  color: #69ade2 !important;
+}
+
+.control-button-green {
+  background: none !important;
+  border-color: #51ad00 !important;
+  border-width: 2px !important;
+  color: #51ad00 !important;
+}
+
+.small-big {
+  font-size: 15pt !important;
+}
+
+.no-label-chatbot > div > div:nth-child(1) {
+  display: none;
+}
+
+#chat-section {
+  position: fixed;
+  align-self: end;
+  width: 65%;
+  z-index: 10000;
+  border: none !important;
+  background: none;
+  padding-left: 0px;
+  padding-right: 0px;
+}
+
+#chat-section > div:nth-child(3) {
+  # background: white;
+}
+
+#chat-section .form {
+  position: relative !important;
+  bottom: 130px;
+  width: 90%;
+  margin: auto;
+  border-radius: 20px;
+}
+
+#chat-section .icon {
+  display: none;
+}
+
+#chat-section .label-wrap {
+  text-align: right;
+  display: block;
+}
+
+#chat-section .label-wrap span {
+  font-size: 30px;
+}
+
+#chat-buttons {
+  position: relative !important;
+  bottom: 130px;
+  width: 90%;
+  margin: auto;
+}
+
+@media only screen and (max-width: 500px) {
+  .main {
+    width: 100% !important;
+    margin: auto;
+  }
+  
+  #chat-section {
+    width: 95%;
+  }
+}
+
+.font-big textarea {
+  font-size: 19pt !important;
+  text-align: center;
+}
+
+.no-label-image-audio > div:nth-child(2) {
+  display: none;
+}
+
+.no-label-radio > span {
+  display: none;
+}
+"""

constants/desc.py

@@ -0,0 +1,17 @@
+pre_phase_description = """
+Zero2Story is a framework built on top of [PaLM API](https://developers.generativeai.google), [Stable Diffusion](https://en.wikipedia.org/wiki/Stable_Diffusion), [MusicGen](https://audiocraft.metademolab.com/musicgen.html) for ordinary people to create their own stories. This framework consists of the **background setup**, **character setup**, and **interative story generation** phases.
+"""
+
+background_setup_phase_description = """
+In this phase, users can setup the genre, place, and mood of the story. Especially, genre is the key that others are depending on. 
+"""
+character_setup_phase_description = """
+In this phase, users can setup characters up to four. For each character, users can decide their characteristics and basic information such as name, age, MBTI, and personality. Also, the image of each character could be generated based on the information using Stable Diffusion. 
+
+PaLM API translates the given character information into a list of keywords that Stable Diffusion could effectively understands. Then, Stable Diffusion generates images using the keywords as a prompt.
+"""
+story_generation_phase_description = """
+In this phase, the first few paragraphs are generated solely based on the information from the background and character setup phases. Afterwards, users could choose a direction from the given three options that PaLM API generated. Then, further stories are generated based on users' choice. This cycle of choosing an option and generating further stories are interatively continued until users decides to stop. 
+
+In each story generation, users also could generate background images and music that describe each scene using Stable Diffusion and MusicGen. If the generated story, options, image, and music in each turn, users could ask to re-generate them.
+"""

constants/init_values.py

@@ -0,0 +1,49 @@
+genres = ["Middle Ages", "Cyberpunk", "Science Fiction", "Horror", "Romance", "Mystery", "Thriller", "Survival", "Post-apocalyptic", "Historical Fiction"]
+
+places = {
+  "Middle Ages": ["Royal Palace", "Small Village", "Enchanted Forest", "Church", "City Walls and Beyond", "Wizard's Tower", "Inn", "Battlefield", "Grand Library", "Royal Gardens"],
+  "Cyberpunk": ["Neon-lit City Streets", "Underground Bar", "Rave Club", "Tech Market", "Hacker Lounge", "Metropolis Central", "Virtual Reality Hub", "Flying Car Docking Station", "Illegal Cybernetic Clinic", "Information Trade Point"],
+  "Science Fiction": ["Space Station", "Futuristic City", "Alien Planet", "Hidden Moon Base", "Cybernetic Hub", "Galactic Headquarters", "Robotics Factory", "Intergalactic Trading Post", "Alien Cultural Center", "Virtual Reality Realm"],
+  "Horror": ["Abandoned House", "Cemetery", "Mental Hospital", "Cathedral", "Forest", "Museum", "Basement", "Abandoned Theme Park", "Abandoned School", "Dark Alley"],
+  "Romance": ["Beach", "Library", "Starlit Bridge", "Lake", "Flower Shop", "Candlelit Restaurant", "Garden", "Cobblestone Alley", "Windy Road", "Ocean View Deck"],
+  "Mystery": ["Haunted House", "Ancient Castle", "Secret Lab", "Dark City Alleyways", "Underground Laboratory", "Historic Art Museum", "Antique Library", "Mythical Ruins", "Modern City Skyscraper", "Deserted Island"],
+  "Thriller": ["Labyrinth", "Abandoned Hospital", "Downtown Alleyway", "Locked Room", "Basement", "Cabin in the Woods", "Abandoned Amusement Park", "Police Station", "Underground Warehouse", "Secret Research Lab"],
+  "Survival": ["Desert", "Forest", "Glacier", "Urban Ruins", "Underwater", "Island", "Mountain Range", "Stormy Ocean", "Wasteland", "Jungle"],
+  "Post-apocalyptic": ["Abandoned City", "Underground Bunker", "Desert Wastelands", "Radioactive Zones", "Ruined Metropolis", "Overgrown Countryside", "Fortified Community", "Lost Library", "Strategic Bridge", "Ghost Town"],
+  "Historical Fiction": ["Castle", "Ancient City", "Countryside", "Temple", "Town Square", "Expedition Base", "Fortress", "Royal Court", "Medieval Market", "Training Ground"]
+}
+
+moods = {
+  "Middle Ages": ["Epic Adventure", "Deep Romance", "Intense Tension", "Mystical and Magical", "Honor and Principle", "Pain and Despair", "Danger and Peril", "Grand Feast and Court Life", "Hope in Darkness", "Traditional National and Cultural"],
+  "Cyberpunk": ["Neon Nights", "Rain-soaked Ambiance", "Electric Energy", "Holographic Illusions", "Cyber Rhythm", "Dark Alley Mysteries", "High-speed Chase", "Augmented Reality Fashion", "Tech-induced Uncertainty", "Tranquility amidst Chaos"],
+  "Science Fiction": ["Technological Advancement", "First Contact", "Galactic Warfare", "Deep Space Exploration", "Intergalactic Romance", "Survival in Space", "Political Intrigue", "Covert Operations", "Interstellar Festival", "Technological Dystopia"],
+  "Horror": ["Ominous", "Mysterious", "Brutal", "Supernatural", "Intense", "Unexpected", "Silent Horror", "Confusing", "Insanity", "Atmospheric Horror"],
+  "Romance": ["Poetic", "Dreamy", "Heartfelt", "Cheerful", "Melancholic", "Innocent", "Exhilarating", "Sweet", "Cozy", "Sunlit"],
+  "Mystery": ["Dark and Gritty", "Silent Suspense", "Time-sensitive Thrill", "Unpredictable Twist", "Momentary Peace", "Unknown Anxiety", "Suspicion and Uncertainty", "Unsettling Atmosphere", "Shocking Revelation", "Loneliness and Isolation"],
+  "Thriller": ["Uneasiness", "Suspicion", "Tension", "Anxiety", "Chase", "Mystery", "Darkness", "Escape", "Secrecy", "Danger"],
+  "Survival": ["Desperate", "Tense", "Adventurous", "Dangerous", "Frightening", "Desolate", "Primitive", "Stealthy", "Stagnant", "Clinical"],
+  "Post-apocalyptic": ["Struggle for Survival", "Beacon of Hope", "Mistrust and Suspicion", "Constant Danger", "Sole Survivor", "Gradual Recovery", "Rebellion Against Oppression", "Pockets of Serenity", "Nature's Emptiness", "Desperate Solidarity"],
+  "Historical Fiction": ["Anticipation", "Awe", "Tranquility", "Tension", "Festive", "Mysterious", "Unexpected", "Focused", "Dichotomy"]
+}
+
+jobs = {
+  "Middle Ages": ["Knight", "Archer", "Wizard/Mage", "Ruler", "Cleric/Priest", "Merchant", "Blacksmith", "Bard", "Barbarian", "Alchemist"],
+  "Cyberpunk": ["Hacker", "Bounty Hunter", "Corporate Executive", "Rebel", "Data Courier", "Cyborg", "Street Mercenary", "Investigative Journalist", "VR Designer", "Virtual Artist"],
+  "Science Fiction": ["Astronaut", "Space Engineer", "Exoplanet Researcher", "Xenobiologist", "Space Bounty Hunter", "Starship Explorer", "AI Developer", "Intergalactic Trader", "Galactic Diplomat", "Virtual Reality Game Developer"],
+  "Horror": ["Doctor", "Detective", "Artist", "Nurse", "Astrologer", "Shaman", "Exorcist", "Journalist", "Scientist", "Gravekeeper"],
+  "Romance": ["Novelist", "Florist", "Barista", "Violinist", "Actor", "Photographer", "Diary Keeper", "Fashion Designer", "Chef", "Traveler"],
+  "Mystery": ["Detective", "Investigative Journalist", "Crime Scene Investigator", "Mystery Novelist", "Defense Attorney", "Psychologist", "Archaeologist", "Secret Agent", "Hacker", "Museum Curator"],
+  "Thriller": ["Detective", "Journalist", "Forensic Scientist", "Hacker", "Police Officer", "Profiler", "Secret Agent", "Security Specialist", "Fraud Investigator", "Criminal Psychologist"],
+  "Survival": ["Explorer", "Marine", "Jungle Guide", "Rescue Worker", "Survivalist", "Mountaineer", "Diver", "Pilot", "Extreme Weather Researcher", "Hunter"],
+  "Post-apocalyptic": ["Scout", "Survivalist", "Archaeologist", "Trader", "Mechanic", "Medical Aid", "Militia Leader", "Craftsman", "Farmer", "Builder"],
+  "Historical Fiction": ["Knight", "Explorer", "Diplomat", "Historian", "General", "Monarch", "Merchant", "Archer", "Landlord", "Priest"]
+}
+
+ages = ["10s", "20s", "30s", "40s", "50s"]
+mbtis = ["ESTJ", "ENTJ", "ESFJ", "ENFJ", "ISTJ", "ISFJ", "INTJ", "INFJ", "ESTP", "ESFP", "ENTP", "ENFP", "ISTP", "ISFP", "INTP", "INFP"]
+random_names = ["Aaron", "Abigail", "Adam", "Adrian", "Alan", "Alexandra", "Alyssa", "Amanda", "Amber", "Amy", "Andrea", "Andrew", "Angela", "Angelina", "Anthony", "Antonio", "Ashley", "Austin", "Benjamin", "Brandon", "Brian", "Brittany", "Brooke", "Bruce", "Bryan", "Caleb", "Cameron", "Carol", "Caroline", "Catherine", "Charles", "Charlotte", "Chase", "Chelsea", "Christopher", "Cody", "Colin", "Connor", "Cooper", "Corey", "Cristian", "Daniel", "David", "Deborah", "Denise", "Dennis", "Derek", "Diana", "Dorothy", "Douglas", "Dylan", "Edward", "Elizabeth", "Emily", "Emma", "Eric", "Ethan", "Evan", "Gabriel", "Gavin", "George", "Gina", "Grace", "Gregory", "Hannah", "Harrison", "Hayden", "Heather", "Helen", "Henry", "Holly", "Hope", "Hunter", "Ian", "Isaac", "Isabella", "Jack", "Jacob", "James", "Jason", "Jeffrey", "Jenna", "Jennifer", "Jessica", "Jesse", "Joan", "John", "Jonathan", "Joseph", "Joshua", "Justin", "Kayla", "Kevin", "Kimberly", "Kyle", "Laura", "Lauren", "Lawrence", "Leah", "Leo", "Leslie", "Levi", "Lewis", "Liam", "Logan", "Lucas", "Lucy", "Luis", "Luke", "Madison", "Maegan", "Maria", "Mark", "Matthew", "Megan", "Michael", "Michelle", "Molly", "Morgan", "Nathan", "Nathaniel", "Nicholas", "Nicole", "Noah", "Olivia", "Owen", "Paige", "Parker", "Patrick", "Paul", "Peter", "Philip", "Phoebe", "Rachel", "Randy", "Rebecca", "Richard", "Robert", "Roger", "Ronald", "Rose", "Russell", "Ryan", "Samantha", "Samuel", "Sandra", "Sarah", "Scott", "Sean", "Sebastian", "Seth", "Shannon", "Shawn", "Shelby", "Sierra", "Simon", "Sophia", "Stephanie", "Stephen", "Steven", "Sue", "Susan", "Sydney", "Taylor", "Teresa", "Thomas", "Tiffany", "Timothy", "Todd", "Tom", "Tommy", "Tracy", "Travis", "Tyler", "Victoria", "Vincent", "Violet", "Warren", "William", "Zach", "Zachary", "Zoe"]
+personalities = ['Optimistic', 'Kind', 'Resilient', 'Generous', 'Humorous', 'Creative', 'Empathetic', 'Ambitious', 'Adventurous']
+
+default_character_images = ["assets/image.png"]
+
+styles = ["sd character", "cartoon", "realistic"]

interfaces/chat_ui.py

@@ -0,0 +1,135 @@
+import gradio as gr
+
+from interfaces import utils
+from modules import palmchat
+
+from pingpong import PingPong
+
+def rollback_last_ui(history):
+    return history[:-1]
+
+async def chat(
+    user_input, chat_mode, chat_state,
+    genre, place, mood,
+    name1, age1, mbti1, personality1, job1,
+    name2, age2, mbti2, personality2, job2,
+    name3, age3, mbti3, personality3, job3,
+    name4, age4, mbti4, personality4, job4,
+    chapter1_title, chapter2_title, chapter3_title, chapter4_title,
+    chapter1_plot, chapter2_plot, chapter3_plot, chapter4_plot
+):
+    chapter_title_ctx = ""
+    if chapter1_title != "":
+        chapter_title_ctx = f"""
+chapter1 {{
+    title: {chapter1_title},
+    plot: {chapter1_plot}
+}}
+
+chapter2 {{
+    title: {chapter2_title},
+    plot: {chapter2_plot}
+}}
+
+chapter3 {{
+    title: {chapter3_title},
+    plot: {chapter3_plot}
+}}
+
+chapter4 {{
+    title: {chapter4_title},
+    plot: {chapter4_plot}
+}}
+"""
+
+    ctx = f"""You are a professional writing advisor, especially specialized in developing ideas on plotting stories and creating characters. I provide genre, where, and mood along with the rough description of one main character and three side characters. 
+
+Give creative but not too long responses based on the following information.
+
+genre: {genre}
+where: {place}
+mood: {mood}
+
+main character: {{
+name: {name1},
+job: {job1},
+age: {age1},
+mbti: {mbti1},
+personality: {personality1} 
+}}
+
+side character1: {{
+name: {name2},
+job: {job2},
+age: {age2},
+mbti: {mbti2},
+personality: {personality2} 
+}}
+
+side character2: {{
+name: {name3},
+job: {job3},
+age: {age3},
+mbti: {mbti3},
+personality: {personality3} 
+}}
+
+side character3: {{
+name: {name4},
+job: {job4},
+age: {age4},
+mbti: {mbti4},
+personality: {personality4} 
+}}
+
+{chapter_title_ctx}
+"""
+
+    ppm = chat_state[chat_mode]
+    ppm.ctx = ctx
+    ppm.add_pingpong(
+        PingPong(user_input, '')
+    )    
+    prompt = utils.build_prompts(ppm)
+
+    response_txt = await utils.get_chat_response(prompt, ctx=ctx)
+    ppm.replace_last_pong(response_txt)
+    
+    chat_state[chat_mode] = ppm
+
+    return (
+        "", 
+        chat_state, 
+        ppm.build_uis(), 
+        gr.update(interactive=True)
+    )
+
+async def chat_regen(chat_mode, chat_state):
+    ppm = chat_state[chat_mode]
+    
+    user_input = ppm.pingpongs[-1].ping
+    ppm.pingpongs = ppm.pingpongs[:-1]
+    ppm.add_pingpong(
+        PingPong(user_input, '')
+    )    
+    prompt = utils.build_prompts(ppm)
+
+    response_txt = await utils.get_chat_response(prompt, ctx=ppm.ctx)
+    ppm.replace_last_pong(response_txt)
+    
+    chat_state[chat_mode] = ppm
+
+    return (
+        chat_state,
+        ppm.build_uis()
+    )
+
+def chat_reset(chat_mode, chat_state):
+    chat_state[chat_mode] = palmchat.GradioPaLMChatPPManager()
+
+    return (
+        "", 
+        chat_state, 
+        [], 
+        gr.update(interactive=False)
+    )

interfaces/plot_gen_ui.py

@@ -0,0 +1,227 @@
+import re
+import gradio as gr
+from interfaces import utils
+from modules import palmchat
+
+def _add_side_character(
+	enable, prompt, cur_side_chars,
+	name, age, mbti, personality, job
+):
+	if enable:
+		prompt = prompt + f"""
+side character #{cur_side_chars}
+- name: {name},
+- job: {job},
+- age: {age},
+- mbti: {mbti},
+- personality: {personality}
+
+"""
+		cur_side_chars = cur_side_chars + 1
+		
+	return prompt, cur_side_chars
+	
+
+async def plot_gen(
+	temperature,
+	genre, place, mood,
+	side_char_enable1, side_char_enable2, side_char_enable3,
+	name1, age1, mbti1, personality1, job1,
+	name2, age2, mbti2, personality2, job2,
+	name3, age3, mbti3, personality3, job3,
+	name4, age4, mbti4, personality4, job4,
+):
+	cur_side_chars = 1
+	prompt = f"""Write a title and an outline of a novel based on the background information below in Ronald Tobias's plot theory. The outline should follow the  "rising action", "crisis", "climax", "falling action", and "denouement" plot types. Each should be filled with a VERY detailed and descriptive at least two paragraphs of string. Randomly choose if the story goes optimistic or tragic.
+
+background information:
+- genre: string
+- where: string
+- mood: string
+
+main character
+- name: string
+- job: string
+- age: string
+- mbti: string
+- personality: string
+
+JSON output:
+{{
+	"title": "string", 
+	"outline": {{
+		"rising action": "paragraphs of string", 
+		"crisis": "paragraphs of string", 
+		"climax": "paragraphs of string", 
+		"falling action": "paragraphs of string", 
+		"denouement": "paragraphs of string"
+	}}
+}}
+
+background information:
+- genre: {genre}
+- where: {place}
+- mood: {mood}
+
+main character
+- name: {name1}
+- job: {job1}
+- age: {age1}
+- mbti: {mbti1}
+- personality: {personality1}
+
+"""
+
+	prompt, cur_side_chars = _add_side_character(
+		side_char_enable1, prompt, cur_side_chars,
+		name2, job2, age2, mbti2, personality2
+	)
+	prompt, cur_side_chars = _add_side_character(
+		side_char_enable2, prompt, cur_side_chars,
+		name3, job3, age3, mbti3, personality3
+	)
+	prompt, cur_side_chars = _add_side_character(
+		side_char_enable3, prompt, cur_side_chars,
+		name4, job4, age4, mbti4, personality4
+	)
+
+	prompt = prompt + "JSON output:\n"
+	
+	print(f"generated prompt:\n{prompt}")
+	parameters = {
+		'model': 'models/text-bison-001',
+		'candidate_count': 1,
+		'temperature': temperature,
+		'top_k': 40,
+		'top_p': 1,
+		'max_output_tokens': 4096,
+	}    	
+	response_json = await utils.retry_until_valid_json(prompt, parameters=parameters)
+
+	return (
+		response_json['title'],
+		f"## {response_json['title']}",
+		response_json['outline']['rising action'],
+		response_json['outline']['crisis'],
+		response_json['outline']['climax'],
+		response_json['outline']['falling action'],
+		response_json['outline']['denouement'],
+	)
+
+
+async def first_story_gen(
+	title, 
+	rising_action, crisis, climax, falling_action, denouement,
+	genre, place, mood,
+	side_char_enable1, side_char_enable2, side_char_enable3,
+	name1, age1, mbti1, personality1, job1,
+	name2, age2, mbti2, personality2, job2,
+	name3, age3, mbti3, personality3, job3,
+	name4, age4, mbti4, personality4, job4, 
+	cursors, cur_cursor
+):
+	cur_side_chars = 1
+	
+	prompt = f"""Write the chapter title and the first few paragraphs of the "rising action" plot based on the background information below in Ronald Tobias's plot theory. Also, suggest three choosable actions to drive current story in different directions. The first few paragraphs should be filled with a VERY MUCH detailed and descriptive at least two paragraphs of string. 
+	
+REMEMBER the first few paragraphs should not end the whole story and allow leaway for the next paragraphs to come. 
+The whole story SHOULD stick to the "rising action -> crisis -> climax -> falling action -> denouement" flow, so REMEMBER not to write anything mentioned from the next plots of crisis, climax, falling action, and denouement yet.
+
+background information:
+- genre: string
+- where: string
+- mood: string
+
+main character
+- name: string
+- job: string
+- age: string
+- mbti: string
+- personality: string
+
+overall outline
+- title: string
+- rising action: string
+- crisis: string
+- climax: string
+- falling action: string
+- denouement: string
+
+JSON output:
+{{
+	"chapter_title": "string",
+	"paragraphs": ["string", "string", ...],
+	"actions": ["string", "string", "string"]
+}}
+
+background information:
+- genre: {genre}
+- where: {place}
+- mood: {mood}
+
+main character
+- name: {name1}
+- job: {job1},
+- age: {age1},
+- mbti: {mbti1},
+- personality: {personality1}
+
+"""
+
+	prompt, cur_side_chars = _add_side_character(
+		side_char_enable1, prompt, cur_side_chars,
+		name2, job2, age2, mbti2, personality2
+	)
+	prompt, cur_side_chars = _add_side_character(
+		side_char_enable2, prompt, cur_side_chars,
+		name3, job3, age3, mbti3, personality3
+	)
+	prompt, cur_side_chars = _add_side_character(
+		side_char_enable3, prompt, cur_side_chars,
+		name4, job4, age4, mbti4, personality4
+	)
+
+	prompt = prompt + f"""
+overall outline
+- title: {title}
+- rising action: {rising_action}
+- crisis: {crisis}
+- climax: {climax}
+- falling action: {falling_action}
+- denouement: {denouement}
+
+JSON output:
+"""
+
+	print(f"generated prompt:\n{prompt}")
+	parameters = {
+		'model': 'models/text-bison-001',
+		'candidate_count': 1,
+		'temperature': 1,
+		'top_k': 40,
+		'top_p': 1,
+		'max_output_tokens': 4096,
+	}    
+	response_json = await utils.retry_until_valid_json(prompt, parameters=parameters)
+
+	chapter_title = response_json["chapter_title"]
+	pattern = r"Chapter\s+\d+\s*[:.]"
+	chapter_title = re.sub(pattern, "", chapter_title)
+
+	cursors.append({
+		"title": chapter_title,
+		"plot_type": "rising action",
+		"story": "\n\n".join(response_json["paragraphs"])
+	})
+
+	return (
+		f"### {chapter_title} (\"rising action\")",
+		"\n\n".join(response_json["paragraphs"]),
+		cursors,
+		cur_cursor, 
+		gr.update(interactive=True),
+		gr.update(interactive=True),
+		gr.update(value=response_json["actions"][0], interactive=True),
+		gr.update(value=response_json["actions"][1], interactive=True),
+		gr.update(value=response_json["actions"][2], interactive=True),
+	)

interfaces/story_gen_ui.py

@@ -0,0 +1,476 @@
+import re
+import copy
+import random
+import gradio as gr
+from gradio_client import Client
+from pathlib import Path
+
+from modules import (
+	ImageMaker, MusicMaker, palmchat, merge_video
+)
+from interfaces import utils
+
+from pingpong import PingPong
+from pingpong.context import CtxLastWindowStrategy
+
+# TODO: Replace checkpoint filename to Huggingface URL
+img_maker = ImageMaker('landscapeAnimePro_v20Inspiration.safetensors', vae="cute20vae.safetensors")
+#img_maker = ImageMaker('fantasyworldFp16.safetensors', vae="cute20vae.safetensors")
+#img_maker = ImageMaker('forgesagalandscapemi.safetensors', vae="anythingFp16.safetensors")
+bgm_maker = MusicMaker(model_size='large', output_format='mp3')
+
+video_gen_client_url = "https://0447df3cf5f7c49c46.gradio.live"
+
+async def update_story_gen(
+	cursors, cur_cursor_idx,
+	genre, place, mood,
+	main_char_name, main_char_age, main_char_mbti, main_char_personality, main_char_job,
+	side_char_enable1, side_char_name1, side_char_age1, side_char_mbti1, side_char_personality1, side_char_job1,
+	side_char_enable2, side_char_name2, side_char_age2, side_char_mbti2, side_char_personality2, side_char_job2,
+	side_char_enable3, side_char_name3, side_char_age3, side_char_mbti3, side_char_personality3, side_char_job3,
+):
+    if len(cursors) == 1:
+        return await first_story_gen(
+			cursors,
+			genre, place, mood,
+			main_char_name, main_char_age, main_char_mbti, main_char_personality, main_char_job,
+			side_char_enable1, side_char_name1, side_char_age1, side_char_mbti1, side_char_personality1, side_char_job1,
+			side_char_enable2, side_char_name2, side_char_age2, side_char_mbti2, side_char_personality2, side_char_job2,
+			side_char_enable3, side_char_name3, side_char_age3, side_char_mbti3, side_char_personality3, side_char_job3,
+			cur_cursor_idx=cur_cursor_idx
+		)
+    else:
+        return await next_story_gen(
+			cursors,
+			None,
+			genre, place, mood,
+			main_char_name, main_char_age, main_char_mbti, main_char_personality, main_char_job,
+			side_char_enable1, side_char_name1, side_char_age1, side_char_mbti1, side_char_personality1, side_char_job1,
+			side_char_enable2, side_char_name2, side_char_age2, side_char_mbti2, side_char_personality2, side_char_job2,
+			side_char_enable3, side_char_name3, side_char_age3, side_char_mbti3, side_char_personality3, side_char_job3,
+			cur_cursor_idx=cur_cursor_idx
+		)
+
+async def next_story_gen(
+	cursors,
+	action,
+	genre, place, mood,
+	main_char_name, main_char_age, main_char_mbti, main_char_personality, main_char_job,
+	side_char_enable1, side_char_name1, side_char_age1, side_char_mbti1, side_char_personality1, side_char_job1,
+	side_char_enable2, side_char_name2, side_char_age2, side_char_mbti2, side_char_personality2, side_char_job2,
+	side_char_enable3, side_char_name3, side_char_age3, side_char_mbti3, side_char_personality3, side_char_job3,	
+	cur_cursor_idx=None
+):
+	stories = ""
+	cur_side_chars = 1
+
+	action = cursors[cur_cursor_idx]["action"] if cur_cursor_idx is not None else action
+	end_idx = len(cursors) if cur_cursor_idx is None else len(cursors)-1
+
+	for cursor in cursors[:end_idx]:
+		stories = stories + cursor["story"]
+
+	prompt = f"""Write the next paragraphs. The next paragraphs should be determined by an option and well connected to the current stories. 
+
+background information:
+- genre: {genre}
+- where: {place}
+- mood: {mood}
+
+main character
+- name: {main_char_name}
+- job: {main_char_job}
+- age: {main_char_age}
+- mbti: {main_char_mbti}
+- personality: {main_char_personality}
+"""
+
+	prompt, cur_side_chars = utils.add_side_character(
+		side_char_enable1, prompt, cur_side_chars,
+		side_char_name1, side_char_job1, side_char_age1, side_char_mbti1, side_char_personality1
+	)
+	prompt, cur_side_chars = utils.add_side_character(
+		side_char_enable2, prompt, cur_side_chars,
+		side_char_name2, side_char_job2, side_char_age2, side_char_mbti2, side_char_personality2
+	)
+	prompt, cur_side_chars = utils.add_side_character(
+		side_char_enable3, prompt, cur_side_chars,
+		side_char_name3, side_char_job3, side_char_age3, side_char_mbti3, side_char_personality3
+	)
+
+	prompt = prompt + f"""
+stories
+{stories}
+
+option to the next stories: {action}
+
+Fill in the following JSON output format:
+{{
+	"paragraphs": "string"
+}}
+
+"""
+
+	print(f"generated prompt:\n{prompt}")
+	parameters = {
+		'model': 'models/text-bison-001',
+		'candidate_count': 1,
+		'temperature': 1.0,
+		'top_k': 40,
+		'top_p': 1,
+		'max_output_tokens': 4096,
+	}    	
+	response_json = await utils.retry_until_valid_json(prompt, parameters=parameters)
+
+	story = response_json["paragraphs"]
+	if isinstance(story, list):
+		story = "\n\n".join(story)
+  
+	if cur_cursor_idx is None:
+		cursors.append({
+			"title": "",
+			"story": story,
+			"action": action
+		})
+	else:
+		cursors[cur_cursor_idx]["story"] = story
+		cursors[cur_cursor_idx]["action"] = action
+
+	return (
+		cursors, len(cursors)-1,
+		story,
+		gr.update(
+			maximum=len(cursors), value=len(cursors),
+			label=f"{len(cursors)} out of {len(cursors)} stories",
+			visible=True, interactive=True
+		),
+		gr.update(interactive=True),
+		gr.update(interactive=True),
+		gr.update(value=None, visible=False, interactive=True),
+		gr.update(value=None, visible=False, interactive=True),
+		gr.update(value=None, visible=False, interactive=True),	
+	)
+
+async def actions_gen(
+	cursors,
+	genre, place, mood,
+	main_char_name, main_char_age, main_char_mbti, main_char_personality, main_char_job,
+	side_char_enable1, side_char_name1, side_char_age1, side_char_mbti1, side_char_personality1, side_char_job1,
+	side_char_enable2, side_char_name2, side_char_age2, side_char_mbti2, side_char_personality2, side_char_job2,
+	side_char_enable3, side_char_name3, side_char_age3, side_char_mbti3, side_char_personality3, side_char_job3,
+	cur_cursor_idx=None
+):
+	stories = ""
+	cur_side_chars = 1
+	end_idx = len(cursors) if cur_cursor_idx is None else len(cursors)-1
+
+	for cursor in cursors[:end_idx]:
+		stories = stories + cursor["story"]
+
+	summary_prompt = f"""Summarize the text below
+
+{stories}
+
+"""
+	print(f"generated prompt:\n{summary_prompt}")
+	parameters = {
+		'model': 'models/text-bison-001',
+		'candidate_count': 1,
+		'temperature': 1.0,
+		'top_k': 40,
+		'top_p': 1,
+		'max_output_tokens': 4096,
+	}    	
+	_, summary = await palmchat.gen_text(summary_prompt, mode="text", parameters=parameters)
+
+	prompt = f"""Suggest the 30 options to drive the stories to the next based on the information below. 
+
+background information:
+- genre: {genre}
+- where: {place}
+- mood: {mood}
+
+main character
+- name: {main_char_name}
+- job: {main_char_job}
+- age: {main_char_age}
+- mbti: {main_char_mbti}
+- personality: {main_char_personality}
+"""
+	prompt, cur_side_chars = utils.add_side_character(
+		side_char_enable1, prompt, cur_side_chars,
+		side_char_name1, side_char_job1, side_char_age1, side_char_mbti1, side_char_personality1
+	)
+	prompt, cur_side_chars = utils.add_side_character(
+		side_char_enable2, prompt, cur_side_chars,
+		side_char_name2, side_char_job2, side_char_age2, side_char_mbti2, side_char_personality2
+	)
+	prompt, cur_side_chars = utils.add_side_character(
+		side_char_enable3, prompt, cur_side_chars,
+		side_char_name3, side_char_job3, side_char_age3, side_char_mbti3, side_char_personality3
+	)
+
+	prompt = prompt + f"""
+summary of the story
+{summary}
+
+Fill in the following JSON output format:
+{{
+    "options": ["string", "string", "string", ...]
+}}
+
+"""
+
+	print(f"generated prompt:\n{prompt}")
+	parameters = {
+		'model': 'models/text-bison-001',
+		'candidate_count': 1,
+		'temperature': 1.0,
+		'top_k': 40,
+		'top_p': 1,
+		'max_output_tokens': 4096,
+	}    	
+	response_json = await utils.retry_until_valid_json(prompt, parameters=parameters)
+	actions = response_json["options"]
+
+	random_actions = random.sample(actions, 3)
+
+	return (
+		gr.update(value=random_actions[0], interactive=True),
+		gr.update(value=random_actions[1], interactive=True),
+		gr.update(value=random_actions[2], interactive=True),
+		"   "
+	)
+
+async def first_story_gen(
+	cursors,
+	genre, place, mood,
+	main_char_name, main_char_age, main_char_mbti, main_char_personality, main_char_job,
+	side_char_enable1, side_char_name1, side_char_age1, side_char_mbti1, side_char_personality1, side_char_job1,
+	side_char_enable2, side_char_name2, side_char_age2, side_char_mbti2, side_char_personality2, side_char_job2,
+	side_char_enable3, side_char_name3, side_char_age3, side_char_mbti3, side_char_personality3, side_char_job3,
+	cur_cursor_idx=None
+):
+	cur_side_chars = 1
+
+	prompt = f"""Write the first three paragraphs of a novel as much detailed as possible. They should be based on the background information. Blend 5W1H principle into the stories as a plain text. Don't let the paragraphs end the whole story.
+
+background information:
+- genre: {genre}
+- where: {place}
+- mood: {mood}
+
+main character
+- name: {main_char_name}
+- job: {main_char_job}
+- age: {main_char_age}
+- mbti: {main_char_mbti}
+- personality: {main_char_personality}
+"""
+
+	prompt, cur_side_chars = utils.add_side_character(
+		side_char_enable1, prompt, cur_side_chars,
+		side_char_name1, side_char_job1, side_char_age1, side_char_mbti1, side_char_personality1
+	)
+	prompt, cur_side_chars = utils.add_side_character(
+		side_char_enable2, prompt, cur_side_chars,
+		side_char_name2, side_char_job2, side_char_age2, side_char_mbti2, side_char_personality2
+	)
+	prompt, cur_side_chars = utils.add_side_character(
+		side_char_enable3, prompt, cur_side_chars,
+		side_char_name3, side_char_job3, side_char_age3, side_char_mbti3, side_char_personality3
+	)
+
+	prompt = prompt + f"""
+Fill in the following JSON output format:
+{{
+	"paragraphs": "string"
+}}
+
+"""	
+
+	print(f"generated prompt:\n{prompt}")
+	parameters = {
+		'model': 'models/text-bison-001',
+		'candidate_count': 1,
+		'temperature': 1.0,
+		'top_k': 40,
+		'top_p': 1,
+		'max_output_tokens': 4096,
+	}    	
+	response_json = await utils.retry_until_valid_json(prompt, parameters=parameters)
+
+	story = response_json["paragraphs"]
+	if isinstance(story, list):
+		story = "\n\n".join(story)
+  
+	if cur_cursor_idx is None:
+		cursors.append({
+			"title": "",
+			"story": story
+		})
+	else:
+		cursors[cur_cursor_idx]["story"] = story
+
+	return (
+		cursors, len(cursors)-1,
+		story,
+		gr.update(
+			maximum=len(cursors), value=len(cursors),
+			label=f"{len(cursors)} out of {len(cursors)} stories",
+			visible=False if len(cursors) == 1 else True, interactive=True
+		),
+		gr.update(interactive=True),
+		gr.update(interactive=True),
+		gr.update(value=None, visible=False, interactive=True),
+		gr.update(value=None, visible=False, interactive=True),	
+  		gr.update(value=None, visible=False, interactive=True),
+	)
+
+def video_gen(
+	image, audio, title, cursors, cur_cursor, use_ffmpeg=True
+):
+	if use_ffmpeg:
+		output_filename = merge_video(image, audio, story_title="")
+	
+	if not use_ffmpeg or not output_filename:
+		client = Client(video_gen_client_url)
+		result = client.predict(
+			"",
+			audio,
+			image,
+			f"{utils.id_generator()}.mp4",
+			api_name="/predict"
+		)
+		output_filename = result[0]
+
+	cursors[cur_cursor]["video"] = output_filename
+
+	return (
+		gr.update(visible=False),
+		gr.update(visible=False),
+		gr.update(visible=True, value=output_filename),
+		cursors,
+		"   "
+	)
+
+
+def image_gen(
+	genre, place, mood, title, story_content, cursors, cur_cursor, story_audio
+):
+	# generate prompts for background image with PaLM
+	for _ in range(3):
+		try:
+			prompt, neg_prompt = img_maker.generate_background_prompts(genre, place, mood, title, "", story_content)
+			neg_prompt
+			print(f"Image Prompt: {prompt}")
+			print(f"Negative Prompt: {neg_prompt}")
+			break
+		except Exception as e:
+			print(e)
+			
+	if not prompt:
+		raise ValueError("Failed to generate prompts for background image.")
+
+	# generate image
+	try:
+		img_filename = img_maker.text2image(prompt, neg_prompt=neg_prompt, ratio='16:9', cfg=6.5)
+	except ValueError as e:
+		print(e)
+		img_filename = str(Path('.') / 'assets' / 'nsfw_warning_wide.png')
+	
+	cursors[cur_cursor]["img"] = img_filename
+
+	video_gen_btn_state = gr.update(interactive=False)
+	if story_audio is not None:
+		video_gen_btn_state = gr.update(interactive=True)
+
+	return  (
+		gr.update(visible=True, value=img_filename),
+		video_gen_btn_state,
+		cursors,
+		"  "
+	)
+
+
+def audio_gen(
+	genre, place, mood, title, story_content, cursors, cur_cursor, story_image
+):
+	# generate prompt for background music with PaLM
+	for _ in range(3):
+		try:
+			prompt = bgm_maker.generate_prompt(genre, place, mood, title, "", story_content)
+			print(f"Music Prompt: {prompt}")
+			break
+		except Exception as e:
+			print(e)
+
+	if not prompt:
+		raise ValueError("Failed to generate prompt for background music.")
+
+	# generate music
+	bgm_filename = bgm_maker.text2music(prompt, length=60)
+	cursors[cur_cursor]["audio"] = bgm_filename
+
+	video_gen_btn_state = gr.update(interactive=False)
+	if story_image is not None:
+		video_gen_btn_state = gr.update(interactive=True)
+
+	return (
+		gr.update(visible=True, value=bgm_filename),
+		video_gen_btn_state,
+		cursors,
+		" "
+	)
+
+def move_story_cursor(moved_cursor, cursors):
+	cursor_content = cursors[moved_cursor-1]
+	max_cursor = len(cursors)
+
+	action_btn = (
+			gr.update(interactive=False),
+			gr.update(interactive=False),
+			gr.update(interactive=False)
+	)
+
+	if moved_cursor == max_cursor:
+		action_btn = (
+			gr.update(interactive=True),
+			gr.update(interactive=True),
+			gr.update(interactive=True)
+		)
+
+	if "video" in cursor_content:
+		outputs = (
+			moved_cursor-1,
+			gr.update(label=f"{moved_cursor} out of {len(cursors)} chapters"),
+			cursor_content["story"],
+			gr.update(value=None, visible=False),
+			gr.update(value=None, visible=False),
+			gr.update(value=cursor_content["video"], visible=True),
+		)
+
+	else:
+		image_container = gr.update(value=None, visible=False)
+		audio_container = gr.update(value=None, visible=False)
+
+		if "img" in cursor_content:
+			image_container = gr.update(value=cursor_content["img"], visible=True)
+
+		if "audio" in cursor_content:
+			audio_container = gr.update(value=cursor_content["audio"], visible=True)
+
+		outputs = (
+			moved_cursor-1,
+			gr.update(label=f"{moved_cursor} out of {len(cursors)} stories"),
+			cursor_content["story"],
+			image_container,
+			audio_container,
+			gr.update(value=None, visible=False),
+		)        
+
+	return outputs + action_btn
+
+def update_story_content(story_content, cursors, cur_cursor):
+	cursors[cur_cursor]["story"] = story_content
+	return cursors

interfaces/ui.py

@@ -0,0 +1,93 @@
+import copy
+import random
+import gradio as gr
+
+import numpy
+import PIL
+from pathlib import Path
+
+from constants.init_values import (
+	places, moods, jobs, random_names, default_character_images
+)
+
+from modules import (
+	ImageMaker, palmchat
+)
+
+from interfaces import utils
+
+# TODO: Replace checkpoint filename to Huggingface URL
+#img_maker = ImageMaker('hellonijicute25d_V10b.safetensors', vae="kl-f8-anime2.vae.safetensors")
+img_maker = ImageMaker('hellonijicute25d_V10b.safetensors') # without_VAE
+
+############
+# for plotting
+
+def get_random_name(cur_char_name, char_name1, char_name2, char_name3):
+	tmp_random_names = copy.deepcopy(random_names)
+	tmp_random_names.remove(cur_char_name)
+	tmp_random_names.remove(char_name1)
+	tmp_random_names.remove(char_name2)
+	tmp_random_names.remove(char_name3)
+	return random.choice(tmp_random_names)
+
+
+def gen_character_image(
+  gallery_images, 
+  name, age, mbti, personality, job, 
+  genre, place, mood, creative_mode
+):
+	# generate prompts for character image with PaLM
+	for _ in range(3):
+		try:
+			prompt, neg_prompt = img_maker.generate_character_prompts(name, age, job, keywords=[mbti, personality, genre, place, mood], creative_mode=creative_mode)
+			print(f"Image Prompt: {prompt}")
+			print(f"Negative Prompt: {neg_prompt}")
+			break
+		except Exception as e:
+			print(e)
+
+	if not prompt:
+		raise ValueError("Failed to generate prompts for character image.")
+
+	# generate image
+	try:
+		img_filename = img_maker.text2image(prompt, neg_prompt=neg_prompt, ratio='3:4', cfg=4.5)
+	except ValueError as e:
+		print(e)
+		img_filename = str(Path('.') / 'assets' / 'nsfw_warning.png')
+
+	# update gallery
+	gen_image = numpy.asarray(PIL.Image.open(img_filename))
+	gallery_images.insert(0, gen_image)
+
+	return gr.update(value=gallery_images), gallery_images
+
+
+def update_on_age(evt: gr.SelectData): 
+	job_list = jobs[evt.value]
+
+	return (
+        gr.update(value=places[evt.value][0], choices=places[evt.value]),
+        gr.update(value=moods[evt.value][0], choices=moods[evt.value]),
+        gr.update(value=job_list[0], choices=job_list),
+        gr.update(value=job_list[0], choices=job_list),
+        gr.update(value=job_list[0], choices=job_list),
+        gr.update(value=job_list[0], choices=job_list)
+	)    
+
+############
+# for tabbing
+
+def update_on_main_tabs(chat_state, evt: gr.SelectData):
+    chat_mode = "plot_chat"
+
+    if evt.value.lower() == "background setup":
+        chat_mode = "plot_chat"
+    elif evt.value.lower() == "story generation":
+        chat_mode = "story_chat"
+    else: # export
+        chat_mode = "export_chat"
+
+    ppm = chat_state[chat_mode]
+    return chat_mode, ppm.build_uis()

interfaces/utils.py

@@ -0,0 +1,80 @@
+import copy
+import json
+import string
+import random
+
+from modules import palmchat
+from pingpong.context import CtxLastWindowStrategy
+
+def add_side_character(
+	enable, prompt, cur_side_chars,
+	name, age, mbti, personality, job
+):
+	if enable:
+		prompt = prompt + f"""
+side character #{cur_side_chars}
+- name: {name},
+- job: {job},
+- age: {age},
+- mbti: {mbti},
+- personality: {personality}
+
+"""
+		cur_side_chars = cur_side_chars + 1
+		
+	return prompt, cur_side_chars
+
+def id_generator(size=6, chars=string.ascii_uppercase + string.digits):
+	return ''.join(random.choice(chars) for _ in range(size))
+
+def parse_first_json_code_snippet(code_snippet):
+	json_parsed_string = None
+	
+	try:
+		json_parsed_string = json.loads(code_snippet, strict=False)
+	except:
+		json_start_index = code_snippet.find('```json')
+		json_end_index = code_snippet.find('```', json_start_index + 6)
+
+		if json_start_index < 0 or json_end_index < 0:
+			raise ValueError('No JSON code snippet found in string.')
+
+		json_code_snippet = code_snippet[json_start_index + 7:json_end_index]
+		json_parsed_string = json.loads(json_code_snippet, strict=False)
+	finally:
+		return json_parsed_string
+
+async def retry_until_valid_json(prompt, parameters=None):
+	response_json = None
+	while response_json is None:
+		_, response_txt = await palmchat.gen_text(prompt, mode="text", parameters=parameters)
+		print(response_txt)
+
+		try:
+			response_json = parse_first_json_code_snippet(response_txt)
+		except:
+			pass
+			
+	return response_json
+
+def build_prompts(ppm, win_size=3):
+	dummy_ppm = copy.deepcopy(ppm)
+	lws = CtxLastWindowStrategy(win_size)
+	return lws(dummy_ppm)
+
+async def get_chat_response(prompt, ctx=None):
+	parameters = {
+		'model': 'models/chat-bison-001',
+		'candidate_count': 1,
+		'context': "" if ctx is None else ctx,
+		'temperature': 1.0,
+		'top_k': 50,
+		'top_p': 0.9,
+	}
+	
+	_, response_txt = await palmchat.gen_text(
+		prompt, 
+		parameters=parameters
+	)
+
+	return response_txt

interfaces/view_change_ui.py

@@ -0,0 +1,13 @@
+import gradio as gr
+    
+def move_to_next_view():
+    return (
+        gr.update(visible=False),
+        gr.update(visible=True),
+    )
+    
+def back_to_previous_view():
+    return (
+        gr.update(visible=True),
+        gr.update(visible=False),
+    )

modules/__init__.py

@@ -0,0 +1,10 @@
+from .image_maker import ImageMaker
+from .music_maker import MusicMaker
+from .palmchat import (
+    PaLMChatPromptFmt,
+    PaLMChatPPManager,
+    GradioPaLMChatPPManager,
+)
+from .utils import (
+    merge_video,
+)

modules/image_maker.py

@@ -0,0 +1,356 @@
+from typing import Literal
+from pathlib import Path
+
+import uuid
+import json
+import re
+import asyncio
+import toml
+
+import torch
+from compel import Compel
+
+from diffusers import (
+    DiffusionPipeline,
+    StableDiffusionPipeline,
+    AutoencoderKL,
+    DPMSolverMultistepScheduler,
+    DDPMScheduler,
+    DPMSolverSinglestepScheduler,
+    DPMSolverSDEScheduler,
+    DEISMultistepScheduler,
+)
+
+from .utils import (
+    set_all_seeds,
+)
+from .palmchat import (
+    palm_prompts,
+    gen_text,
+)
+
+_gpus = 0
+
+class ImageMaker:
+    # TODO: DocString...
+    """Class for generating images from prompts."""
+
+    __ratio = {'3:2':  [768, 512],
+               '4:3':  [680, 512],
+               '16:9': [912, 512],
+               '1:1':  [512, 512],
+               '9:16': [512, 912],
+               '3:4':  [512, 680],
+               '2:3':  [512, 768]}
+    __allocated = False
+
+    def __init__(self, model_base: str,
+                       clip_skip: int = 2,
+                       sampling: Literal['sde-dpmsolver++'] = 'sde-dpmsolver++',
+                       vae: str = None,
+                       safety: bool = True,
+                       neg_prompt: str = None,
+                       device: str = None) -> None:
+        """Initialize the ImageMaker class.
+
+        Args:
+            model_base (str): Filename of the model base.
+            clip_skip (int, optional): Number of layers to skip in the clip model. Defaults to 2.
+            sampling (Literal['sde-dpmsolver++'], optional): Sampling method. Defaults to 'sde-dpmsolver++'.
+            vae (str, optional): Filename of the VAE model. Defaults to None.
+            safety (bool, optional): Whether to use the safety checker. Defaults to True.
+            device (str, optional): Device to use for the model. Defaults to None.
+        """
+
+        self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') if not device else device
+        self.__model_base = model_base
+        self.__clip_skip = clip_skip
+        self.__sampling = sampling
+        self.__vae = vae
+        self.__safety = safety
+        self.neg_prompt = neg_prompt
+
+        print("Loading the Stable Diffusion model into memory...")
+        self.__sd_model = StableDiffusionPipeline.from_single_file(self.model_base,
+                                                              #torch_dtype=torch.float16,
+                                                              use_safetensors=True)
+
+        # Clip Skip
+        self.__sd_model.text_encoder.text_model.encoder.layers = self.__sd_model.text_encoder.text_model.encoder.layers[:12 - (self.clip_skip - 1)]
+
+        # Sampling method
+        if True: # TODO: Sampling method :: self.sampling == 'sde-dpmsolver++'
+            scheduler = DPMSolverMultistepScheduler.from_config(self.__sd_model.scheduler.config)
+            scheduler.config.algorithm_type = 'sde-dpmsolver++'
+            self.__sd_model.scheduler = scheduler
+        
+        # TODO: Use LoRA
+
+        # VAE
+        if self.vae:
+            vae_model = AutoencoderKL.from_single_file(self.vae)
+            self.__sd_model.vae = vae_model
+
+        if not self.safety:
+            self.__sd_model.safety_checker = None
+            self.__sd_model.requires_safety_checker = False
+
+        print(f"Loaded model to {self.device}")
+        self.__sd_model = self.__sd_model.to(self.device)
+
+        # Text Encoder using Compel
+        self.__compel_proc = Compel(tokenizer=self.__sd_model.tokenizer, text_encoder=self.__sd_model.text_encoder, truncate_long_prompts=False)
+        
+        output_dir = Path('.') / 'outputs'
+        if not output_dir.exists():
+            output_dir.mkdir(parents=True, exist_ok=True)
+        elif output_dir.is_file():
+            assert False, f"A file with the same name as the desired directory ('{str(output_dir)}') already exists."
+
+    
+    def text2image(self,
+                   prompt: str, neg_prompt: str = None,
+                   ratio: Literal['3:2', '4:3', '16:9', '1:1', '9:16', '3:4', '2:3'] = '1:1',
+                   step: int = 28,
+                   cfg: float = 4.5,
+                   seed: int = None) -> str:
+        """Generate an image from the prompt.
+
+        Args:
+            prompt (str): Prompt for the image generation.
+            neg_prompt (str, optional): Negative prompt for the image generation. Defaults to None.
+            ratio (Literal['3:2', '4:3', '16:9', '1:1', '9:16', '3:4', '2:3'], optional): Ratio of the generated image. Defaults to '1:1'.
+            step (int, optional): Number of iterations for the diffusion. Defaults to 20.
+            cfg (float, optional): Configuration for the diffusion. Defaults to 7.5.
+            seed (int, optional): Seed for the random number generator. Defaults to None.
+
+        Returns:
+            str: Path to the generated image.
+        """
+
+        output_filename = Path('.') / 'outputs' / str(uuid.uuid4())
+
+        if not seed or seed == -1:
+            seed = torch.randint(0, 2**32 - 1, (1,)).item()
+        set_all_seeds(seed)
+
+        width, height = self.__ratio[ratio]
+
+        prompt_embeds, negative_prompt_embeds = self.__get_pipeline_embeds(prompt, neg_prompt or self.neg_prompt)
+        
+        # Generate the image
+        result = self.__sd_model(prompt_embeds=prompt_embeds,
+                              negative_prompt_embeds=negative_prompt_embeds,
+                              guidance_scale=cfg,
+                              num_inference_steps=step,
+                              width=width,
+                              height=height,
+                            )
+        if self.__safety and result.nsfw_content_detected[0]:
+            print("=== NSFW Content Detected ===")
+            raise ValueError("Potential NSFW content was detected in one or more images.")
+
+        img = result.images[0]
+        img.save(str(output_filename.with_suffix('.png')))
+
+        return str(output_filename.with_suffix('.png'))
+    
+
+    def generate_character_prompts(self, character_name: str, age: str, job: str,
+                                         keywords: list[str] = None, 
+                                         creative_mode: Literal['sd character', 'cartoon', 'realistic'] = 'cartoon') -> tuple[str, str]:
+        """Generate positive and negative prompts for a character based on given attributes.
+
+        Args:
+            character_name (str): Character's name.
+            age (str): Age of the character.
+            job (str): The profession or job of the character.
+            keywords (list[str]): List of descriptive words for the character.
+
+        Returns:
+            tuple[str, str]: A tuple of positive and negative prompts.
+        """
+
+        positive = "" # add static prompt for character if needed (e.g. "chibi, cute, anime")
+        negative = palm_prompts['image_gen']['neg_prompt']
+
+        # Generate prompts with PaLM
+        t = palm_prompts['image_gen']['character']['gen_prompt']
+        q = palm_prompts['image_gen']['character']['query']
+        query_string = t.format(input=q.format(character_name=character_name,
+                                               job=job,
+                                               age=age,
+                                               keywords=', '.join(keywords) if keywords else 'Nothing'))
+        try:
+            response, response_txt = asyncio.run(asyncio.wait_for(
+                                                    gen_text(query_string, mode="text", use_filter=False),
+                                                    timeout=10)
+                                                )
+        except asyncio.TimeoutError:
+            raise TimeoutError("The response time for PaLM API exceeded the limit.")
+        
+        try: 
+            res_json = json.loads(response_txt)
+            positive = (res_json['primary_sentence'] if not positive else f"{positive}, {res_json['primary_sentence']}") + ", "
+            gender_keywords = ['1man', '1woman', '1boy', '1girl', '1male', '1female', '1gentleman', '1lady']
+            positive += ', '.join([w if w not in gender_keywords else w + '+++' for w in res_json['descriptors']])
+            positive = f'{job.lower()}+'.join(positive.split(job.lower()))
+        except:
+            print("=== PaLM Response ===")
+            print(response.filters)
+            print(response_txt)
+            print("=== PaLM Response ===")            
+            raise ValueError("The response from PaLM API is not in the expected format.")
+            
+        return (positive.lower(), negative.lower())
+
+
+    def generate_background_prompts(self, genre:str, place:str, mood:str,
+                                          title:str, chapter_title:str, chapter_plot:str) -> tuple[str, str]:
+        """Generate positive and negative prompts for a background image based on given attributes.
+
+        Args:
+            genre (str): Genre of the story.
+            place (str): Place of the story.
+            mood (str): Mood of the story.
+            title (str): Title of the story.
+            chapter_title (str): Title of the chapter.
+            chapter_plot (str): Plot of the chapter.
+
+        Returns:
+            tuple[str, str]: A tuple of positive and negative prompts.
+        """
+
+        positive = "painting+++, anime+, catoon, watercolor, wallpaper, text---" # add static prompt for background if needed (e.g. "chibi, cute, anime")
+        negative = "realistic, human, character, people, photograph, 3d render, blurry, grayscale, oversaturated, " + palm_prompts['image_gen']['neg_prompt']
+
+        # Generate prompts with PaLM
+        t = palm_prompts['image_gen']['background']['gen_prompt']
+        q = palm_prompts['image_gen']['background']['query']
+        query_string = t.format(input=q.format(genre=genre,
+                                               place=place,
+                                               mood=mood,
+                                               title=title,
+                                               chapter_title=chapter_title,
+                                               chapter_plot=chapter_plot))
+        try:
+            response, response_txt = asyncio.run(asyncio.wait_for(
+                                                    gen_text(query_string, mode="text", use_filter=False),
+                                                    timeout=10)
+                                                )
+        except asyncio.TimeoutError:
+            raise TimeoutError("The response time for PaLM API exceeded the limit.")
+        
+        try: 
+            res_json = json.loads(response_txt)
+            positive = (res_json['main_sentence'] if not positive else f"{positive}, {res_json['main_sentence']}") + ", "
+            positive += ', '.join(res_json['descriptors'])
+        except:
+            print("=== PaLM Response ===")
+            print(response.filters)
+            print(response_txt)
+            print("=== PaLM Response ===")            
+            raise ValueError("The response from PaLM API is not in the expected format.")
+            
+        return (positive.lower(), negative.lower())
+
+
+    def __get_pipeline_embeds(self, prompt:str, negative_prompt:str) -> tuple[torch.Tensor, torch.Tensor]:
+        """
+        Get pipeline embeds for prompts bigger than the maxlength of the pipeline
+
+        Args:
+            prompt (str): Prompt for the image generation.
+            neg_prompt (str): Negative prompt for the image generation.
+
+        Returns:
+            tuple[torch.Tensor, torch.Tensor]: A tuple of positive and negative prompt embeds.
+        """
+        conditioning = self.__compel_proc.build_conditioning_tensor(prompt)
+        negative_conditioning = self.__compel_proc.build_conditioning_tensor(negative_prompt)
+        return self.__compel_proc.pad_conditioning_tensors_to_same_length([conditioning, negative_conditioning])
+
+
+    @property
+    def model_base(self):
+        """Model base
+
+        Returns:
+            str: The model base (read-only)
+        """
+        return self.__model_base
+
+    @property
+    def clip_skip(self):
+        """Clip Skip
+
+        Returns:
+            int: The number of layers to skip in the clip model (read-only)
+        """
+        return self.__clip_skip
+
+    @property
+    def sampling(self):
+        """Sampling method
+
+        Returns:
+            Literal['sde-dpmsolver++']: The sampling method (read-only)
+        """
+        return self.__sampling
+
+    @property
+    def vae(self):
+        """VAE
+
+        Returns:
+            str: The VAE (read-only)
+        """
+        return self.__vae
+
+    @property
+    def safety(self):
+        """Safety checker
+
+        Returns:
+            bool: Whether to use the safety checker (read-only)
+        """
+        return self.__safety
+    
+    @property
+    def device(self):
+        """Device
+
+        Returns:
+            str: The device (read-only)
+        """
+        return self.__device
+
+    @device.setter
+    def device(self, value):
+        if self.__allocated:
+            raise RuntimeError("Cannot change device after the model is loaded.")
+
+        if value == 'cpu':
+            self.__device = value
+        else:
+            global _gpus
+            self.__device = f'{value}:{_gpus}'
+            max_gpu = torch.cuda.device_count()
+            _gpus = (_gpus + 1) if (_gpus + 1) < max_gpu else 0
+        self.__allocated = True
+
+    @property
+    def neg_prompt(self):
+        """Negative prompt
+
+        Returns:
+            str: The negative prompt
+        """
+        return self.__neg_prompt
+
+    @neg_prompt.setter
+    def neg_prompt(self, value):
+        if not value:
+            self.__neg_prompt = ""
+        else:
+            self.__neg_prompt = value

modules/music_maker.py

@@ -0,0 +1,165 @@
+from typing import Literal
+from tempfile import NamedTemporaryFile
+from pathlib import Path
+
+import uuid
+import shutil
+import json
+import asyncio
+import toml
+
+import torch
+
+from audiocraft.models import MusicGen
+from audiocraft.data.audio import audio_write
+from pydub import AudioSegment
+
+from .utils import (
+    set_all_seeds,
+)
+from .palmchat import (
+    palm_prompts,
+    gen_text,
+)
+
+class MusicMaker:
+    # TODO: DocString...
+    """Class for generating music from prompts."""
+
+    def __init__(self, model_size: Literal['small', 'medium', 'melody', 'large'] = 'large',
+                       output_format: Literal['wav', 'mp3'] = 'mp3',
+                       device: str = None) -> None:
+        """Initialize the MusicMaker class.
+
+        Args:
+            model_size (Literal['small', 'medium', 'melody', 'large'], optional): Model size. Defaults to 'large'.
+            output_format (Literal['wav', 'mp3'], optional): Output format. Defaults to 'mp3'.
+            device (str, optional): Device to use for the model. Defaults to None.
+        """
+
+        self.__model_size = model_size
+        self.__output_format = output_format
+        self.__device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') if not device else device
+
+        print("Loading the MusicGen model into memory...")
+        self.__mg_model = MusicGen.get_pretrained(self.model_size, device=self.device)
+        self.__mg_model.set_generation_params(use_sampling=True,
+                                            top_k=250,
+                                            top_p=0.0,
+                                            temperature=1.0,
+                                            cfg_coef=3.0
+                                            )
+        
+        output_dir = Path('.') / 'outputs'
+        if not output_dir.exists():
+            output_dir.mkdir(parents=True, exist_ok=True)
+        elif output_dir.is_file():
+            assert False, f"A file with the same name as the desired directory ('{str(output_dir)}') already exists."
+    
+
+    def text2music(self, prompt: str, length: int = 60, seed: int = None) -> str:
+        """Generate a music from the prompt.
+
+        Args:
+            prompt (str): Prompt to generate the music from.
+            length (int, optional): Length of the music in seconds. Defaults to 60.
+            seed (int, optional): Seed to use for the generation. Defaults to None.
+
+        Returns:
+            str: Path to the generated music.
+        """
+
+        def wavToMp3(src_file: str, dest_file: str) -> None:
+            sound = AudioSegment.from_wav(src_file)  
+            sound.export(dest_file, format="mp3")
+        
+        output_filename = Path('.') / 'outputs' / str(uuid.uuid4())
+
+        if not seed or seed == -1:
+            seed = torch.randint(0, 2**32 - 1, (1,)).item()
+        set_all_seeds(seed)
+
+        self.__mg_model.set_generation_params(duration=length)
+        output = self.__mg_model.generate(descriptions=[prompt], progress=True)[0]
+
+        with NamedTemporaryFile("wb", delete=True) as temp_file:
+            audio_write(temp_file.name, output.cpu(), self.__mg_model.sample_rate, strategy="loudness", loudness_compressor=True)
+            if self.output_format == 'mp3':
+                wavToMp3(f'{temp_file.name}.wav', str(output_filename.with_suffix('.mp3')))
+            else:
+                shutil.copy(f'{temp_file.name}.wav', str(output_filename.with_suffix('.wav')))
+
+        return str(output_filename.with_suffix('.mp3' if self.output_format == 'mp3' else '.wav'))
+
+
+    def generate_prompt(self, genre:str, place:str, mood:str,
+                              title:str, chapter_title:str, chapter_plot:str) -> str:
+        """Generate a prompt for a background music based on given attributes.
+
+        Args:
+            genre (str): Genre of the story.
+            place (str): Place of the story.
+            mood (str): Mood of the story.
+            title (str): Title of the story.
+            chapter_title (str): Title of the chapter.
+            chapter_plot (str): Plot of the chapter.
+
+        Returns:
+            str: Generated prompt.
+        """
+
+        # Generate prompts with PaLM
+        t = palm_prompts['music_gen']['gen_prompt']
+        q = palm_prompts['music_gen']['query']
+        query_string = t.format(input=q.format(genre=genre,
+                                               place=place,
+                                               mood=mood,
+                                               title=title,
+                                               chapter_title=chapter_title,
+                                               chapter_plot=chapter_plot))
+        try:
+            response, response_txt = asyncio.run(asyncio.wait_for(
+                                                    gen_text(query_string, mode="text", use_filter=False),
+                                                    timeout=10)
+                                                )
+        except asyncio.TimeoutError:
+            raise TimeoutError("The response time for PaLM API exceeded the limit.")
+        
+        try: 
+            res_json = json.loads(response_txt)
+        except:
+            print("=== PaLM Response ===")
+            print(response.filters)
+            print(response_txt)
+            print("=== PaLM Response ===")            
+            raise ValueError("The response from PaLM API is not in the expected format.")
+            
+        return res_json['main_sentence']
+
+
+    @property
+    def model_size(self):
+        """Model size
+
+        Returns:
+            Literal['small', 'medium', 'melody', 'large']: The model size (read-only)
+        """
+        return self.__model_size
+
+    @property
+    def output_format(self):
+        """Output format
+
+        Returns:
+            Literal['wav', 'mp3']: The output format (read-only)
+        """
+        return self.__output_format
+
+    @property
+    def device(self):
+        """Device
+
+        Returns:
+            str: The device (read-only)
+        """
+        return self.__device

modules/palmchat.py

@@ -0,0 +1,133 @@
+import os
+import toml
+from pathlib import Path
+import google.generativeai as palm_api
+
+from pingpong import PingPong
+from pingpong.pingpong import PPManager
+from pingpong.pingpong import PromptFmt
+from pingpong.pingpong import UIFmt
+from pingpong.gradio import GradioChatUIFmt
+
+from .utils import set_palm_api_key
+
+
+# Set PaLM API Key
+set_palm_api_key()
+
+# Load PaLM Prompt Templates
+palm_prompts = toml.load(Path('.') / 'assets' / 'palm_prompts.toml')
+
+class PaLMChatPromptFmt(PromptFmt):
+    @classmethod
+    def ctx(cls, context):
+        pass
+
+    @classmethod
+    def prompt(cls, pingpong, truncate_size):
+        ping = pingpong.ping[:truncate_size]
+        pong = pingpong.pong
+        
+        if pong is None or pong.strip() == "":
+            return [
+                {
+                    "author": "USER",
+                    "content": ping
+                },
+            ]
+        else:
+            pong = pong[:truncate_size]
+
+            return [
+                {
+                    "author": "USER",
+                    "content": ping
+                },
+                {
+                    "author": "AI",
+                    "content": pong
+                },
+            ]
+
+class PaLMChatPPManager(PPManager):
+    def build_prompts(self, from_idx: int=0, to_idx: int=-1, fmt: PromptFmt=PaLMChatPromptFmt, truncate_size: int=None):
+        results = []
+        
+        if to_idx == -1 or to_idx >= len(self.pingpongs):
+            to_idx = len(self.pingpongs)
+
+        for idx, pingpong in enumerate(self.pingpongs[from_idx:to_idx]):
+            results += fmt.prompt(pingpong, truncate_size=truncate_size)
+
+        return results    
+
+class GradioPaLMChatPPManager(PaLMChatPPManager):
+    def build_uis(self, from_idx: int=0, to_idx: int=-1, fmt: UIFmt=GradioChatUIFmt):
+        if to_idx == -1 or to_idx >= len(self.pingpongs):
+            to_idx = len(self.pingpongs)
+
+        results = []
+
+        for pingpong in self.pingpongs[from_idx:to_idx]:
+            results.append(fmt.ui(pingpong))
+
+        return results    
+
+async def gen_text(
+    prompt,
+    mode="chat", #chat or text
+    parameters=None,
+    use_filter=True
+):
+    if parameters is None:
+        temperature = 1.0
+        top_k = 40
+        top_p = 0.95
+        max_output_tokens = 1024
+        
+        # default safety settings
+        safety_settings = [{"category":"HARM_CATEGORY_DEROGATORY","threshold":1},
+                           {"category":"HARM_CATEGORY_TOXICITY","threshold":1},
+                           {"category":"HARM_CATEGORY_VIOLENCE","threshold":2},
+                           {"category":"HARM_CATEGORY_SEXUAL","threshold":2},
+                           {"category":"HARM_CATEGORY_MEDICAL","threshold":2},
+                           {"category":"HARM_CATEGORY_DANGEROUS","threshold":2}]
+        if not use_filter:
+            for idx, _ in enumerate(safety_settings):
+                safety_settings[idx]['threshold'] = 4
+
+        if mode == "chat":
+            parameters = {
+                'model': 'models/chat-bison-001',
+                'candidate_count': 1,
+                'context': "",
+                'temperature': temperature,
+                'top_k': top_k,
+                'top_p': top_p,
+            }
+        else:
+            parameters = {
+                'model': 'models/text-bison-001',
+                'candidate_count': 1,
+                'temperature': temperature,
+                'top_k': top_k,
+                'top_p': top_p,
+                'max_output_tokens': max_output_tokens,
+                'safety_settings': safety_settings,
+            }
+
+    if mode == "chat":
+        response = await palm_api.chat_async(**parameters, messages=prompt)
+    else:
+        response = palm_api.generate_text(**parameters, prompt=prompt)
+    
+    if use_filter and len(response.filters) > 0 and \
+        response.filters[0]['reason'] == 2:
+        response_txt = "your request is blocked for some reasons"
+    else:
+        if mode == "chat":
+            response_txt = response.last
+        else:
+            response_txt = response.result
+    
+    return response, response_txt

modules/utils.py

@@ -0,0 +1,109 @@
+import os
+import numpy as np
+import random
+import uuid
+
+from pathlib import Path
+from tempfile import NamedTemporaryFile
+
+from PIL import Image
+from PIL import ImageDraw
+from PIL import ImageFont
+
+import torch
+
+import google.generativeai as palm_api
+
+def set_all_seeds(random_seed: int) -> None:
+    # TODO: DocString...
+    torch.manual_seed(random_seed)
+    torch.cuda.manual_seed(random_seed)
+    torch.cuda.manual_seed_all(random_seed)
+    torch.backends.cudnn.deterministic = True
+    torch.backends.cudnn.benchmark = False
+    np.random.seed(random_seed)
+    random.seed(random_seed)
+    print(f"Using seed {random_seed}")
+
+
+def get_palm_api_key() -> str:
+    palm_api_key = os.getenv("PALM_API_KEY")
+
+    if palm_api_key is None:
+        with open('.palm_api_key.txt', 'r') as file:
+            palm_api_key = file.read().strip()
+
+    if not palm_api_key:
+        raise ValueError("PaLM API Key is missing.")
+    return palm_api_key
+
+
+def set_palm_api_key(palm_api_key:str = None) -> None:
+    palm_api.configure(api_key=(palm_api_key or get_palm_api_key()))
+
+
+def merge_video(image_path: str, audio_path: str, story_title:str = None) -> str:
+    output_filename = Path('.') / 'outputs' / str(uuid.uuid4())
+    output_filename = str(output_filename.with_suffix('.mp4'))
+
+    try:
+        temp_image_path = image_path
+        if story_title:
+            img = Image.open(image_path)
+            img_drawable = ImageDraw.Draw(img)
+            title_font_path = str(Path('.') / 'assets' / 'Lugrasimo-Regular.ttf')
+            title_font = ImageFont.truetype(title_font_path, 24)
+            img_drawable.text((65, 468), story_title, font=title_font, fill=(16, 16, 16))
+            img_drawable.text((63, 466), story_title, font=title_font, fill=(255, 255, 255))
+            
+            with NamedTemporaryFile("wb", delete=True) as temp_file:
+                temp_image_path = f'{temp_file.name}.png'
+                img.save(temp_image_path)
+
+        cmd = [
+            'ffmpeg', '-loop', '1', '-i', temp_image_path, '-i', audio_path,
+            '-filter_complex',
+            '"[1:a]asplit=29[ASPLIT01][ASPLIT02][ASPLIT03][ASPLIT04][ASPLIT05][ASPLIT06][ASPLIT07][ASPLIT08][ASPLIT09][ASPLIT10][ASPLIT11][ASPLIT12][ASPLIT13][ASPLIT14][ASPLIT15][ASPLIT16][ASPLIT17][ASPLIT18][ASPLIT19][ASPLIT20][ASPLIT21][ASPLIT22][ASPLIT23][ASPLIT24][ASPLIT25][ASPLIT26][ASPLIT27][ASPLIT28][ASPLIT29];\
+[ASPLIT01]bandpass=frequency=20:width=4:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ01];\
+[ASPLIT02]bandpass=frequency=25:width=4:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ02];\
+[ASPLIT03]bandpass=frequency=31.5:width=8:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ03];\
+[ASPLIT04]bandpass=frequency=40:width=8:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ04];\
+[ASPLIT05]bandpass=frequency=50:width=8:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ05];\
+[ASPLIT06]bandpass=frequency=63:width=8:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ06];\
+[ASPLIT07]bandpass=frequency=80:width=16:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ07];\
+[ASPLIT08]bandpass=frequency=100:width=16:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ08];\
+[ASPLIT09]bandpass=frequency=125:width=32:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ09];\
+[ASPLIT10]bandpass=frequency=160:width=32:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ10];\
+[ASPLIT11]bandpass=frequency=200:width=64:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ11];\
+[ASPLIT12]bandpass=frequency=250:width=64:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ12];\
+[ASPLIT13]bandpass=frequency=315:width=64:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ13];\
+[ASPLIT14]bandpass=frequency=400:width=64:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ14];\
+[ASPLIT15]bandpass=frequency=500:width=128:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ15];\
+[ASPLIT16]bandpass=frequency=630:width=128:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ16];\
+[ASPLIT17]bandpass=frequency=800:width=128:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ17];\
+[ASPLIT18]bandpass=frequency=1000:width=128:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ18];\
+[ASPLIT19]bandpass=frequency=1250:width=256:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ19];\
+[ASPLIT20]bandpass=frequency=1500:width=256:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ20];\
+[ASPLIT21]bandpass=frequency=2000:width=512:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ21];\
+[ASPLIT22]bandpass=frequency=2500:width=1024:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ22];\
+[ASPLIT23]bandpass=frequency=3150:width=1024:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ23];\
+[ASPLIT24]bandpass=frequency=4000:width=1024:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ24];\
+[ASPLIT25]bandpass=frequency=5000:width=1024:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ25];\
+[ASPLIT26]bandpass=frequency=6300:width=1024:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ26];\
+[ASPLIT27]bandpass=frequency=8000:width=1024:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ27];\
+[ASPLIT28]bandpass=frequency=12000:width=1024:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ28];\
+[ASPLIT29]bandpass=frequency=16000:width=2048:width_type=h,showvolume=rate=30.000:c=0xAFFFFFFF:b=5:w=176:h=11:o=v:t=0:v=0:m=p:s=0:ds=lin:dm=1:dmc=0xFFFFFFFF[EQ29];\
+[EQ01][EQ02][EQ03][EQ04][EQ05][EQ06][EQ07][EQ08][EQ09][EQ10][EQ11][EQ12][EQ13][EQ14][EQ15][EQ16][EQ17][EQ18][EQ19][EQ20][EQ21][EQ22][EQ23][EQ24][EQ25][EQ26][EQ27][EQ28][EQ29]hstack=inputs=29[BARS];[0][BARS]overlay=(W-w)/2:H-h-50:shortest=1,format=yuv420p[out]"',
+            '-map', '"[out]"', '-map', '1:a', '-movflags', '+faststart',
+            output_filename
+        ]
+
+        result = os.system(' '.join([c.strip() for c in cmd]))
+
+        if result == 0:
+            return output_filename
+        else:
+            return None
+    except Exception as e:
+        print(e)
+        return None

pyproject.toml

@@ -0,0 +1,36 @@
+[tool.poetry]
+name = "zero2story"
+version = "0.1.0"
+description = ""
+authors = ["Sangjoon Han <[email protected]>"]
+readme = "README.md"
+
+[tool.poetry.dependencies]
+python = ">=3.10,<3.13"
+gradio = "^3.42.0"
+torch = {version = "^2.0.1+cu118", source = "pytorch"}
+torchvision = {version = "^0.15.2+cu118", source = "pytorch"}
+torchaudio = {version = "^2.0.2+cu118", source = "pytorch"}
+transformers = "^4.33.1"
+scipy = "^1.11.2"
+diffusers = "^0.20.2"
+numpy = ">=1.21,<1.25"
+numba = "^0.57.1"
+audiocraft = "^0.0.2"
+accelerate = "^0.22.0"
+google-generativeai = "^0.1.0"
+bingbong = "^0.4.2"
+asyncio = "^3.4.3"
+toml = "^0.10.2"
+compel = "^2.0.2"
+
+[[tool.poetry.source]]
+name = "pytorch"
+url = "https://download.pytorch.org/whl/cu118"
+priority = "explicit"
+
+[tool.poetry.group.dev.dependencies]
+
+[build-system]
+requires = ["poetry-core"]
+build-backend = "poetry.core.masonry.api"

run.sh

@@ -0,0 +1,19 @@
+#!/bin/bash
+
+PID=./gradio.pid
+if [[ -f "$PID" ]]; then
+    kill -15 `cat $PID` || kill -9 `cat $PID`
+fi
+
+mkdir -p ./logs
+rm -rf ./logs/app.log
+
+CONFIDENTIAL=./.palm_api_key.txt
+if [[ ! -f "$CONFIDENTIAL" ]]; then
+    echo "Error: PaLM API file not found. To continue, please create a .palm_api_key.txt file in the current directory."
+    exit 1
+fi
+
+export PALM_API_KEY=`cat .palm_api_key.txt`
+nohup python -u app.py > ./logs/app.log 2>&1 &
+echo $! > $PID