WaledRashed24/comics

1---2title: AI Comic Factory3emoji: 👩‍🎨4colorFrom: red5colorTo: yellow6sdk: docker7pinned: true8app_port: 30009disable_embedding: false10short_description: Create your own AI comic with a single prompt11hf_oauth: true12hf_oauth_expiration_minutes: 4320013hf_oauth_scopes: [inference-api]14---15 16# AI Comic Factory17 18Last release: AI Comic Factory 1.219 20The AI Comic Factory will soon have an official website: [aicomicfactory.app](https://aicomicfactory.app)21 22For more information about my other projects please check [linktr.ee/FLNGR](https://linktr.ee/FLNGR).23 24## Running the project at home25 26First, I would like to highlight that everything is open-source (see [here](https://huggingface.co/spaces/jbilcke-hf/ai-comic-factory/tree/main), [here](https://huggingface.co/spaces/jbilcke-hf/VideoChain-API/tree/main), [here](https://huggingface.co/spaces/hysts/SD-XL/tree/main), [here](https://github.com/huggingface/text-generation-inference)).27 28However the project isn't a monolithic Space that can be duplicated and ran immediately:29it requires various components to run for the frontend, backend, LLM, SDXL etc.30 31If you try to duplicate the project, open the `.env` you will see it requires some variables.32 33Provider config:34- `LLM_ENGINE`: can be one of `INFERENCE_API`, `INFERENCE_ENDPOINT`, `OPENAI`, `GROQ`, `ANTHROPIC`35- `RENDERING_ENGINE`: can be one of: "INFERENCE_API", "INFERENCE_ENDPOINT", "REPLICATE", "VIDEOCHAIN", "OPENAI" for now, unless you code your custom solution36 37Auth config:38- `AUTH_HF_API_TOKEN`:  if you decide to use Hugging Face for the LLM engine (inference api model or a custom inference endpoint)39- `AUTH_OPENAI_API_KEY`: to use OpenAI for the LLM engine40- `AUTH_GROQ_API_KEY`: to use Groq for the LLM engine41- `AUTH_ANTHROPIC_API_KEY`: to use Anthropic (Claude) for the LLM engine42- `AUTH_VIDEOCHAIN_API_TOKEN`: secret token to access the VideoChain API server43- `AUTH_REPLICATE_API_TOKEN`: in case you want to use Replicate.com44 45Rendering config:46- `RENDERING_HF_INFERENCE_ENDPOINT_URL`: necessary if you decide to use a custom inference endpoint47- `RENDERING_REPLICATE_API_MODEL_VERSION`: url to the VideoChain API server48- `RENDERING_HF_INFERENCE_ENDPOINT_URL`: optional, default to nothing49- `RENDERING_HF_INFERENCE_API_BASE_MODEL`: optional, defaults to "stabilityai/stable-diffusion-xl-base-1.0"50- `RENDERING_HF_INFERENCE_API_REFINER_MODEL`: optional, defaults to "stabilityai/stable-diffusion-xl-refiner-1.0"51- `RENDERING_REPLICATE_API_MODEL`: optional, defaults to "stabilityai/sdxl"52- `RENDERING_REPLICATE_API_MODEL_VERSION`: optional, in case you want to change the version53 54Language model config (depending on the LLM engine you decide to use):55- `LLM_HF_INFERENCE_ENDPOINT_URL`: "<use your own>"56- `LLM_HF_INFERENCE_API_MODEL`: "HuggingFaceH4/zephyr-7b-beta"57- `LLM_OPENAI_API_BASE_URL`: "https://api.openai.com/v1"58- `LLM_OPENAI_API_MODEL`: "gpt-4-turbo"59- `LLM_GROQ_API_MODEL`: "mixtral-8x7b-32768"60- `LLM_ANTHROPIC_API_MODEL`: "claude-3-opus-20240229"61 62In addition, there are some community sharing variables that you can just ignore.63Those variables are not required to run the AI Comic Factory on your own website or computer64(they are meant to create a connection with the Hugging Face community,65and thus only make sense for official Hugging Face apps):66- `NEXT_PUBLIC_ENABLE_COMMUNITY_SHARING`: you don't need this67- `COMMUNITY_API_URL`: you don't need this68- `COMMUNITY_API_TOKEN`: you don't need this69- `COMMUNITY_API_ID`: you don't need this70 71Please read the `.env` default config file for more informations.72To customise a variable locally, you should create a `.env.local`73(do not commit this file as it will contain your secrets).74 75-> If you intend to run it with local, cloud-hosted and/or proprietary models **you are going to need to code 👨‍💻**.76 77## The LLM API (Large Language Model)78 79Currently the AI Comic Factory uses [zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) through an [Inference Endpoint](https://huggingface.co/docs/inference-endpoints/index).80 81You have multiple options:82 83### Option 1: Use an Inference API model84 85This is a new option added recently, where you can use one of the models from the Hugging Face Hub. By default we suggest to use [zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) as it will provide better results than the 7b model.86 87To activate it, create a `.env.local` configuration file:88 89```bash90LLM_ENGINE="INFERENCE_API"91 92HF_API_TOKEN="Your Hugging Face token"93 94# "HuggingFaceH4/zephyr-7b-beta" is used by default, but you can change this95# note: You should use a model able to generate JSON responses,96# so it is storngly suggested to use at least the 34b model97HF_INFERENCE_API_MODEL="HuggingFaceH4/zephyr-7b-beta"98```99 100### Option 2: Use an Inference Endpoint URL101 102If you would like to run the AI Comic Factory on a private LLM running on the Hugging Face Inference Endpoint service, create a `.env.local` configuration file:103 104```bash105LLM_ENGINE="INFERENCE_ENDPOINT"106 107HF_API_TOKEN="Your Hugging Face token"108 109HF_INFERENCE_ENDPOINT_URL="path to your inference endpoint url"110```111 112To run this kind of LLM locally, you can use [TGI](https://github.com/huggingface/text-generation-inference) (Please read [this post](https://github.com/huggingface/text-generation-inference/issues/726) for more information about the licensing).113 114### Option 3: Use an OpenAI API Key115 116This is a new option added recently, where you can use OpenAI API with an OpenAI API Key.117 118To activate it, create a `.env.local` configuration file:119 120```bash121LLM_ENGINE="OPENAI"122 123# default openai api base url is: https://api.openai.com/v1124LLM_OPENAI_API_BASE_URL="A custom OpenAI API Base URL if you have some special privileges"125 126LLM_OPENAI_API_MODEL="gpt-4-turbo"127 128AUTH_OPENAI_API_KEY="Yourown OpenAI API Key"129```130### Option 4: (new, experimental) use Groq131 132```bash133LLM_ENGINE="GROQ"134 135LLM_GROQ_API_MODEL="mixtral-8x7b-32768"136 137AUTH_GROQ_API_KEY="Your own GROQ API Key"138```139### Option 5: (new, experimental) use Anthropic (Claude)140 141```bash142LLM_ENGINE="ANTHROPIC"143 144LLM_ANTHROPIC_API_MODEL="claude-3-opus-20240229"145 146AUTH_ANTHROPIC_API_KEY="Your own ANTHROPIC API Key"147```148 149### Option 6: Fork and modify the code to use a different LLM system150 151Another option could be to disable the LLM completely and replace it with another LLM protocol and/or provider (eg. Claude, Replicate), or a human-generated story instead (by returning mock or static data).152 153### Notes154 155It is possible that I modify the AI Comic Factory to make it easier in the future (eg. add support for Claude or Replicate)156 157## The Rendering API158 159This API is used to generate the panel images. This is an API I created for my various projects at Hugging Face.160 161I haven't written documentation for it yet, but basically it is "just a wrapper ™" around other existing APIs:162 163- The [hysts/SD-XL](https://huggingface.co/spaces/hysts/SD-XL?duplicate=true) Space by [@hysts](https://huggingface.co/hysts)164- And other APIs for making videos, adding audio etc.. but you won't need them for the AI Comic Factory165 166### Option 1: Deploy VideoChain yourself167 168You will have to [clone](https://huggingface.co/spaces/jbilcke-hf/VideoChain-API?duplicate=true) the [source-code](https://huggingface.co/spaces/jbilcke-hf/VideoChain-API/tree/main)169 170Unfortunately, I haven't had the time to write the documentation for VideoChain yet.171(When I do I will update this document to point to the VideoChain's README)172 173 174### Option 2: Use Replicate175 176To use Replicate, create a `.env.local` configuration file:177 178```bash179RENDERING_ENGINE="REPLICATE"180 181RENDERING_REPLICATE_API_MODEL="stabilityai/sdxl"182 183RENDERING_REPLICATE_API_MODEL_VERSION="da77bc59ee60423279fd632efb4795ab731d9e3ca9705ef3341091fb989b7eaf"184 185AUTH_REPLICATE_API_TOKEN="Your Replicate token"186```187 188### Option 3: Use another SDXL API189 190If you fork the project you will be able to modify the code to use the Stable Diffusion technology of your choice (local, open-source, proprietary, your custom HF Space etc).191 192It would even be something else, such as Dall-E.193