Hello there, Prompt Muse here! In this comprehensive guide, I’ll walk you through the essentials of setting up ComfyUI and Animate Diff Evolve.

1. Introduction

• ComfyUI offers a node-based layout, allowing for a streamlined workspace tailored to your needs.mm
• Drag and drop features for images and workflows enhance ease of use.
• This tutorial aims to make you feel comfortable with ComfyUI, ensuring you make the most of its unique features.

2. System Requirements

• Nvidia RTX Graphics card is preferable (Works with AMD cards and Macs click here)
• Preferred: 12 GB of VRAM for processing videos.
• Allocate 50-80 GB of storage for outputs.

Tip: If your PC doesn’t meet these requirements, consider using Shadow Tech. It’s a remote PC service I personally use, providing an NVIDIA 1080 GPU, ample storage, and 12 GB VRAM for about $40/month.

3. Software Dependencies

• Git: Allows you to pull extensions from GitHub. Download here.
• FFmpeg: Essential for encoding videos. Download here.

4. Installing ComfyUI & comfyUIManager

1. Visit the ComfyUI GitHub page.
2. Under the “Installing” section, select the “Direct link to download” for the standalone portable version.
3. Once downloaded, extract the files to your chosen directory.
4. For ComfyUI models:
• Checkpoints: Download from civitai.com and place in the checkpoints folder.
• VAE: Download from Stability AI’s hugging face website and place in the vae folder.
5. Install ComfyUI Manager from the civitAI page for easy updates and add-on installations.

5. Basic Workflow Overview

• ComfyUI operates like a circuit board, with nodes representing each process.
• Start with the Load Checkpoints node, input your positive and negative prompts, and proceed to the K Sampler.
• The Latent Image Node determines your image dimensions and batch size.
• The VAE Decode node processes the final image.

Pro Tip: Images created in ComfyUI can be dragged and dropped back into the system to load their respective node layouts.

Here’s the continuation of the tutorial based on the provided transcript:

—

## **6. Installing Motion Models**

1. In ComfyUI Manager, go to **Install Models**.
2. Type “mm” in the search box, which stands for motion models.
3. Download and install **Motion Model version 14**, **version 15**, and **version 15 uncore 2**. The high and mid variants are also recommended.
4. After downloading, you should see a confirmation message.

## **7. Installing Control Net**

1. Still in the ComfyUI Manager, search for “control net”.
2. Locate and install **Stable Diffusion 1.5**. You might need to scroll down the list.
3. Install other control nets like **Line Artarts** and **Open Pose** as needed, ensuring you have enough storage.

**Note:** After installing, close all instances of the manager and ComfyUI. When reopening ComfyUI, it will automatically install the nodes and extensions you’ve selected.

## **8. Utilizing Pre-Made Templates**

1. Visit **cinka dink’s GitHub page**. He offers pre-made templates for Animate Evolve.
2. Download a workflow, drag, and drop it into ComfyUI to load the node layout.
3. If you encounter missing nodes (highlighted in red), simply go to the manager and click on **Install Missing Nodes**.

## **9. Running Workflows**

1. Use the **Load Checkpoints** node, followed by **Clip Setting**, **Positive Prompt**, and **Negative Prompt**.
2. The **K Sampler** determines sampling, while **Animate Diff Loader** adds motion.
3. Adjust settings such as image dimensions, batch size, and motion model as needed.
4. For advanced users, experiment with nodes like **Animate Diff Uniform Context Options** to enhance animations.

## **10. Adding Laura for Enhanced Details**

1. Double-click on an empty space in ComfyUI and search for **Laura**.
2. Select the **Laur Loader** and connect it as shown in the workflow.
3. Adjust the **Strength Model** to reduce the risk of artifacts.
4. Run the prompt and observe the enhanced details in the generated animation.

## **11. Video to Video Animations Workflow**

1. Use **Inner Reflections’** guide as a template for video-to-video animations.
2. For video inputs, use the **Video Load Node**.
3. Adjust settings like frame load cap, skip first frames, and select every M frame to customize the video output.
4. Use **Load Checkpoint** to select your desired model.
5. Implement control nets, like **Line Art**, to guide the animation style.
6. Experiment with different nodes and settings to achieve unique video outputs.

## **12. Exploring Other Templates**

1. Another great template to explore is **Prompt Scheduling**, which allows dynamic prompt changes over time.
2. This feature can be used to create evolving animations, adding depth to your projects.

—

**Conclusion**

Thank you for following along! With ComfyUI, the possibilities are vast, allowing for creative freedom and intricate animations. A written version of this tutorial is available on the Prompt Muse website. Feel free to reach out on social media @promptMuse for further assistance.

Remember: Art is a journey, and every masterpiece begins with the decision to try. Dive in and explore the world of ComfyUI and Animate Evolve!

<p>The post ComfyUI and Animate Diff Evolve Installation Guide first appeared on Prompt Muse.</p>

Getting Started with Warp Fusion

Prerequisites

Before we dive into the tutorial, there are a few prerequisites you need to have:

• Google Colab Pro or Google Colab Pro Plus
• Access to Alex’s Patreon page where you can get the Warp Fusion notebook
• A model and a Lora from Civic AI

Setting Up Warp Fusion

The first step in using Warp Fusion is setting up the environment. This involves downloading the Warp Fusion notebook from Alex’s Patreon page and loading it into your Google Colab. Once you have the notebook ready, you need to connect it to a hosted runtime if you’re using Google Colab Pro. This ensures that you’re using your compute unit and the GPU.

Configuring Warp Fusion

Basic Settings

Once your environment is set up, you can start configuring Warp Fusion. The first thing you need to do is specify the name of the folder where your output files will be stored in your Google Drive. You also need to set the width and height of your output video to match your input video. A resolution of 720 by 1280 is a good starting point, but you can adjust this to suit your needs.

Video Input Settings

Next, you need to specify the path to your input video. This is the video that you want to animate using Warp Fusion. You can save your input video on your Google Drive for easy access. If your video is long and you want to reduce the diffusion time, you can set the ‘extra frame’ setting to two, which means Warp Fusion will diffuse every other frame.

Video Masking

Video masking is a useful feature that can help make the background of your animation consistent and stable. You can use your input video as the mask source and extract the background mask. If you want to add another video into the background, you can specify the path to that video in the ‘mask video path’ setting.

Defining SD and K Functions

The next step is to define the SD and K functions. You need to change the ‘load to’ setting to GPU and specify the path to the model you want to use. You can get models from Civic AI or Hugging Face. You also need to specify the directory where your control net models will be stored on your Google Drive.

Running Warp Fusion

Once you have all your settings configured, you can start running Warp Fusion. This involves running all the cells in the notebook up to the GUI section. This process can take about 5 to 10 minutes, and you will get a green tick for every cell that has successfully completed.

Customizing Your Animation

Using Prompts and Loras

Prompts and Loras are powerful features that can help you customize your animation. Prompts are instructions that guide the AI in creating the animation, while Loras are elements that you can add to your animation. You can specify your prompts and Loras in the GUI section of the notebook. You can also adjust the strength of the stylization and the prompt guidance to achieve the desired effect.

Using Control Nets

Control nets are models that you can use to control the animation. You can select the control net models you want to use and adjust their weights to influence their impact on the animation.

Using the Warp Tab

The warp taballows you to adjust the flow blend of your animation. This is the blending of your input video with the next frame of stylization. If you find that your animation is over-stylized, you can reduce the flow blend to achieve a more balanced effect.

Using the Mask Tab

The mask tab allows you to use a background mask for your animation. You can change the color or use an image or the original video as the background. This can help to create a more consistent and stable background for your animation.

Rendering Your Animation

Once you have customized your animation, you can start rendering it. This involves running the ‘diffuse’ cell in the notebook. As your animation progresses, you will see a preview frame that allows you to check for any errors. If there are errors or things you want to fix, you can stop the diffusion and adjust your settings.

Creating a Video from Your Animation

After your animation has been rendered, you can create a video from it by running the ‘Create Video’ cell. This will create a video from the frames that were diffused in the previous step. The video and all the frames will be saved to your Google Drive.

Post-Production

After you have created your video, you can bring the frames into a post-production software like DaVinci Resolve or After Effects for further editing. This can involve reducing the flickering of the animation or exporting the frames into a .mov file.

Upscaling Your Video

The final step in the process is upscaling your video. This can be done using a service like Topaz Labs or Pixel. These services use AI to increase the size of the video and make it sharper. Pixel is a browser-based service that is easy to use and offers a pay-as-you-go pricing model, making it a cost-effective alternative to Topaz Labs.

Warp Fusion is a powerful tool that allows you to create unique and eye-catching animations. With its wide range of features and customization options, you can create animations that truly stand out. Whether you’re a hobbyist or a professional, Warp Fusion offers a fun and innovative way to create animations.

Remember to share your creations on social media and tag Prompt Muse. We love to see what you create with Warp Fusion. Happy prompting!

<p>The post Warp Fusion: A Comprehensive Step-by-Step Tutorial first appeared on Prompt Muse.</p>

Artificial Intelligence (AI) has rapidly become one of the most versatile and powerful technological tools in recent years, providing users with a wealth of opportunities to explore, create, and express themselves in new and exciting ways.

In the world of music, AI has revolutionized the way musicians create, record, and market their art. From optimizing vocal tracks to generating entirely new compositions, AI is making music more accessible, dynamic, and personalized than ever before.
In this blog post, we will explore the many ways in which AI is transforming the world of music and the exciting possibilities it holds for the future.

The Rise of AI Vocals

One of the most fascinating developments in the music industry has been the rise of AI vocals, which has enabled musicians to create realistic vocal tracks without requiring the services of a professional singer. With AI vocals, musicians can fine-tune the pitch, timbre, and other vocal qualities to suit their creative vision, ensuring the final track sounds exactly as they imagined it.

AI vocals are also useful for people who might not have the self-confidence to sing themselves or who lack the resources to hire a professional singer. By providing a flexible and affordable solution, AI vocals allow artists to experiment with different styles, sounds, and arrangements without breaking the bank.

Collaborating with AI

The ability to collaborate with AI has also opened up new creative avenues for musicians, allowing them to create brand new vocals by combining different AI-generated voices. This makes it possible to create choirs without requiring a large number of singers, providing musicians with greater control over the final product.
In addition, AI can help improve the quality of music videos that independent artists produce, enabling them to create high-quality visuals without the need for expensive equipment or professional crews. This allows musicians to maintain their creative vision while still producing music videos that are visually stunning and engaging for their fans.

Learning from Others

The podcast team at Prompt Muse often discusses their creative process, which involves brainstorming ideas together for days or weeks, testing concepts, and then starting production in a relaxed and casual way. They also rely on hard work and spend hours researching information to learn new things, including the latest AI techniques.

To stay ahead of the curve, they often check Reddit for 10 minutes a day to keep up with the latest technology and advancements. Although some may find AI technology daunting, the team at Prompt Muse believes in adapting their workflow to new technology as it evolves. They remain open-minded and learn by experimenting and transforming assets to achieve their objectives quickly.

Misconceptions About AI

There are many misconceptions about AI, particularly its ability to create videos and music easily and independently. While AI can be used as a tool for creating quick visualizers, human input is still necessary for creating a good story and continuity. Music videos are seen as short films with a soundtrack and require a lot of time and effort to produce.

By suggesting that AI can replace the human touch, creators often find themselves feeling frustrated. Dismissing their work as “AI video” or “AI music” doesn’t recognize the human input and effort involved in the creative process.

AI-assisted Music Videos

However, AI can be a great ally for independent music artists in producing their own videos and music. In particular, Runway.ai is a popular tool used by the podcast team to edit backgrounds that are then added to the video edit on Adobe After Effects. While it is important to present the reality of what happened on the day of the shoot, it is equally important to bring the artist’s creativity and vision to life.
AI-generated music still requires the artist to record the song from start to finish – AI only changes the tone of their voice to match someone else – but previous tracks can be used to create a new track. Working in tandem with AI also allows the artist to produce something of which they are proud.

The Future of AI in Music

As AI technology continues to evolve, the future of music promises to be more exciting than ever before. With AI, artists will be able to explore new creative frontiers while also reaching new and diverse audiences around the world.

AI-generated music may not only be used by independent artists, but also by established stars to collaborate with other artists and explore new musical styles. This could help to expand the boundaries of music as we know it while also creating new and unique sounds that have never been heard before.

Innovation is the key to the future of music and Prompt Muse is committed to bringing our readers the latest developments in AI and music. The podcast and blog team encourages their audience to embrace the power of AI as a creative tool to develop their own unique vision.
As AI technology continues to evolve, it’s exciting to think about what the future will hold for music. With AI, musicians will be able to create more exciting, innovative, and personalized music than ever before, empowering them to take their creativity to new heights and explore new possibilities in the world of music.

FAQ

1. What are some benefits of AI vocals for artists and producers?
– AI vocals can be useful for people who are not self-confident enough to sing, and they can be merged to create brand new vocals and create choirs without needing many people.
2. How can AI assist independent music artists in creating high-quality music videos?
– AI can give power to music artists to create videos and music on their own, faster than before, and it can be a great ally for independent music artists in producing their own videos and music.
3. How do the guests on the podcast approach video production and what tools do they use?
– They start with walking in a Japanese environment, then edit the video to add a background frame, play with camera movements, and transform assets using Enringing; they use Warp fusion for primary tools and Deform for making scenes more interesting, and Erased backgrounds are edited on Runway.
4. Are there any misconceptions about AI and its ability to create videos and music?
– Yes, some people believe that AI can create videos and music with just one sentence or prompt, but human input is still necessary for creating a good story and continuity.
5. How do independent music artists feel about the use of AI in their work?
– There is frustration among creators when people dismiss their work as “AI video” or “AI music” without recognizing the human input and effort involved.
6. How has AI technology evolved and improved over time?
– When the speaker started with AI technology, there were limited resources available, but now tools like Dalle Fusion are available and powerful, allowing for greater creativity and faster problem-solving.
7. What tools and websites are available to bring creative ideas to life?
– There are many websites and tools available to create 3D visuals and animate drawings, and Runway can be used to achieve dream boosting and generating unique ideas.
8. What is Sagans, and what have they accomplished through their work with AI music videos?
– Sagans is an anonymous AI collective that has been producing music videos for popular artists since 2022, and they have produced videos for Lincoln Park Lawns Entropy and Die Antwood’s Age of Illusion in just one year.
9. How important is it to stay up to date with AI advancements as a creative?
– It is important to be aware of new technology for future use, but it is not necessary to know everything, and it is important to check casually without feeling overwhelmed.
10. How can creativity be enhanced through setting limits and restrictions?
– James Blake created a song by limiting himself to only four notes, and the N-word style and technique were developed mainly through experimentation and prompt.

<p>The post Sagans: The Anonymous AI Collective Taking Over the Music Video World first appeared on Prompt Muse.</p>

Workflow Overview

The workflow we will be following involves using mocap data to animate the characters, which can be obtained for free or created yourself using an iPhone or a mocap suit. We will then use the iClone Character Creator software to create an actor and add the mocap data to it. The final step involves using the Automatic 111 Web UI with Six Hunter’s Python script and ControlNet enabled to generate the animations.

To achieve the desired results, we will also be using a Scarlett Johansson trained Laura file and a diffusion checkpoint file. You can train your own files or choose from thousands of available models and textual conversions for your project.

Step 1: Create a Base Character

First, create a base character that doesn’t need to look exactly like your final character but should have similar features such as hair, clothing style, and physique. The face can be overwritten with the AI, but it’s helpful to have similar features as a guide for the AI. The iClone Character Creator software is a great tool for creating characters, as it allows you to easily drag and drop hair, clothing, and other elements onto your character.

Step 2: Obtain Mocap Data

Next, obtain mocap data for your character’s movements. You can create your own by recording a video of yourself and uploading it to websites like Plask and Deep Motion, which will export an FBX file for free. Simply drag and drop the exported FBX file onto your character in iClone to apply the mocap data.

Step 3: Animate the Character

Once your character has the mocap data applied, you can begin animating them. There are multiple ways to do this, such as using facial rigging controlled by your phone or AI-generated voice and lip-syncing. When you’re happy with the animation, render the frames as a sequence, which will be used later for stable diffusion.

Step 4: Set Up Automatic 111 Web UI and ControlNet

Next, set up the Automatic 111 Web UI and ControlNet by installing the required files and scripts, including Six Hunter’s Python script, the Scarlett Johansson Laura file, and the diffusion checkpoint file. Make sure to enable ControlNet and allow other scripts to control the extension in the settings.

Step 5: Generate the Animation

With everything set up, you can now generate the animation using the Automatic 111 Web UI. Start by uploading the first frame of your animation, which will guide the rest of the animation. Adjust settings such as denoising strength, sampling method, and ControlNet model to achieve the desired results. Once you’re happy with the generated frame, lock in the seed to ensure consistency in all the images.

Step 6: Apply the Script and Render the Animation

Finally, apply Six Hunter’s multi-frame video rendering script to generate the animation. Upload your guide frames, enable color correction, and choose the input frame loopback source. After generating the animation, you can use software like After Effects or DaVinci Resolve to compile the frames and apply post-production effects such as deflickering and motion blur.

Conclusion

Creating coherent AI animations can be a complex process, but with the right tools and workflow, you can achieve impressive results. By using mocap data, iClone Character Creator, and Six Hunter’s Python script, you can have complete control over your characters and their actions. This workflow allows for endless possibilities in creating unique and engaging animations for your projects.

Transcript:

Today, I’ve got a very exciting workflow. I’m going to be showing you how to create coherent AI animations without all the glitching all over the place. Before we start, I must say that this workflow would not happen without Zampious aka Six Hunter. He has created the script that I’m using to produce these results. Now, I have worked out a workflow to use around the script to get really nice results that you can control every aspect of. To be honest, using stock footage is pretty useless because you have absolutely zero control on what that person is doing. Of course, you can get a green screen and train a model to look like yourself and act the part. But that makes absolutely no sense because then you have to be hiring a team of actresses and actors. In this video, I’m going to be showing you how to have ultimate control of your characters, what they look like, and what they look like they do. Before we start the step by step process of what I did to achieve these animations, I’m going to give you an overview of the workflow. With that said, let’s hop into the workflow overview.

For those who have guessed it on my Twitter and Instagram and other socials that I was using mocap data, you are correct. There are several places you can get mocap data from. So you can pick up mocap data for free or you can make it yourself on your iPhone. Granted, that’s not the best way to do it. The best way to do it is to use a mocap suit, but they are rather expensive. So I simply just mix and blend my purchased mocap data in with my facial rig that I control with my phone, and it’s easily compiled together in iCologne by character creator. The next step is to create an actor and just add that mocap data. The actor is essentially telling the AI what loose styles we want, so it doesn’t have to look exactly like your end result, just a guide for the AI. The last part is to use automatic 111 Web UI with 6 hunks Python script enabled and control net enabled as well. And I show you exactly how to do that. I’m also using, can you guess, a Scarlett Johansson trained Laura file, if you haven’t noticed that already, as well as a diffusion checkpoint file.

I’m actually using both of them and I’ll show you which ones I use. You can always train your own to create your own styles, but there is thousands and thousands of models and textual conversions and hyper networks and lords being created all the time. So you can look through the list and see which one you like. And if you’re using for a commercial project, I suggest you make your own. The first step is to create a base of our character. Now, this doesn’t need to look exactly like your character. You just need to keep the features such as the hair, the style of the clothing, and their physique. Now, the face can be overwritten with the AI, but it’s quite good to keep similar features. Although, again, like I say, it doesn’t have to be exactly the same. It’s just something there to guide and help the AI. I absolutely love character creator for just dragging and dropping hair on and have stylisation, add clothes. I can add additional clothes via blender. I can buy them from the marketplace. The same with the hair, you can use the hair that is in the system. You can change it, you can make it longer, and everything is built into a pipeline, which I absolutely love.

Obviously, it comes at a cost, but for me, it’s worth it. The second step is to get your mocap data, which essentially is your actor’s movement, what they’re going to do. You can make your own by making a video of yourself and uploading it to websites like Plask and Deep Motion. What they will do is export your FBX file for free. I simply drag and drop for that exported FBX file onto my character in iC loan, and then start with making the character talk. Now, there are multiple ways you can do this. You can use their face where you can use their actual lip, which is really cool because you can use AI on top of it to change your voice. Once I’m happy with the animation, I can either send it to unreal blender and put cameras in and record it from there. But to be honest, I don’t even do that at this point. I just go to render and render video and go to image and then set it as sequence and then my output size to what I want to use. And then I render and this takes about two minutes to render the frames ready for stable diffusion.

I get a lot of questions about what GPU I have. You don’t want my computer, I actually use a cloud GPU, which in non technical turns is a remote computer. I actually just connect, sign in and use my stable diffusion template on there. So it’s very easy to use. You can see my prior video on how to set it up and install. In this video, I’m going to be using ControlNet, so you’ll need to have that installed. That’s also in my previous video, so it shows you my complete setup from there and how I use it. Anyway, on with the video. I just log into my Run pod. Now, if you’re using a local version of Automatic 111, this will work for you because the file structures are exactly the same. There’s a couple of things that we need to set up before we get started. We’ve got to make sure we’ve got to make sure we got a checkpoint file. So I’m going to load this into the back end of my stable diffusion. To do this, I’m just going to connect to my pod. So I’m going to connect to the JupyterLab, which is my file structure that runs my automatic 111.

This is the same as the file structure that you find on your local version of stable diffusion. As you can see, if I click on the stable diffusion file, you’ll probably recognize this file structure as is that is the same as your local one. And that’s why you can follow along if you’re doing this locally on your PC. So the first thing we want to do is go to models and then Stable Diffusion. You need to put your checkpoint folder in here. And as you can see, I’m using realistic vision. Ckpt. And if we go over here to Civet AI, you can see this is where I downloaded it from. So you can download it here on your local version. If you’re using Run pod, just press this down arrow and model safe tensor. Right click on that, copy link, come back to your JupyterLab and click on Terminal down here and just simply type in, we get space control V for paste and hit return on the keyboard. And that will ultimately download the file. Now we use the We get protocol because it’s the quickest way to download a large file onto Run pod.

You can download it locally to your machine and drag and drop it into a file structure. But it can take sometimes a long time to do that, especially if you’ve got slow internet connection. So this is my preferable way. If it’s a small file, I usually just download it locally to my machine and drag it across. But checkpoint files tend to be quite big. See there, it’s downloaded and it’s given it a horrible name without an extension. So we’re going to hit F2 on a keyboard and name it. Ckpt. I’m naming it Silly because I’ve already got that file. So now that’s done, we can close down this terminal window and we want to now load in a Lo ra file. So if you don’t know what a Lo ra file is, it gives you the stylisation on top of your checkpoint file. So I’m going to be using these in combination in this video. Now, I feel like I get quite good results from doing this. We go up the hierarchy back to Models and then go to Lo ra. So you should see Lo ra. We were just in that folder there. So click on Lo ra.

And if you ever lost to where I am, just look at this path here and it will tell you exactly where I am. We go back to Civet AI. So I’m going to be using the Scarlett Johansson Lo ra file. I’m not going to take her complete likeness. I’m just going to take pinch just for this tutorial to show you, you can actually create your own Lo ra styles. And if you want me to do a tutorial on that, this is a really small file. It’s 144 megabytes. You can just download that locally to your machine and then come over to JupyterLab. And if I go to Downloads, you can see I’ve got it here. You just drag and drop it into there. The last thing we need to do in this back end is import our script. So if you press this button again to go up the hierarchy and come to script. So give that a click. And we are going to be using 6hunt script in here. If you go to xanthias. H. I o, this is where you can download your file. If you can afford it, please donate to him. He’s given the stable diffusion and AI community loads and helped out massively.

But if you can’t afford it, you can press no thanks and download it for free, which is really generous of him. And that will download that to your local machine. Once it has downloaded, come back to your downloads and simply drag and drop again into your run pod. So you should see it there, multi frame, underscore, render. Yeah, we have done everything we need to in the back end. So now we need to look at the front end, which is the automatic 111 Web UI. So we come back to my pod, we’re going to now connect to the stable diffusion Web UI, which is the interface. Again, this is the same as your local version. So we’re going to head directly to the Settings tab up here and then come down here to Control net. There’s a couple of things here we just want to check. You can add multiple Control net models to one render. I’m going to call it render for now, I think. And I’ve got two. I’m only going to use one model, but I wanted to show you this here. So you can put all nine if you really wanted to.

But my favourite is actually Canny, so I’m just going to be using one. If you come down here, you need to allow other scripts to control this extension. This is vital. You need this to be checked. I repeat, you need this to be checked. Once that’s all done, we click on Apply settings and then click on Reload UI. You’re going to get a bad gateway. Do not panic. This is normal. Do not worry. So we’re going to just close that window and come back to my pods and click on your Connect to HTTP. And again, this will reload your web UI with all your settings done. Now, let’s get to the fun bit. So I’m going to go to image to image, which is this tab here. And if you see this little sun here, I think it’s a sun, under the generate button, give that a click. And then we’re going to go to the Laura tab. Here you can see your Laura styles. So like I said before, we’re going to be using the Scarlett Johansson. Now, if you don’t see your Laura file in here, you can give it a refresh. That doesn’t work.

I advise you to come back out of your pods, click this hamburger icon and restart. Please do not restart. You need to restart your pod. And that will take a minute to restart everything. And that should refresh everything in here. I’m going to click on the Scarlett Johansson. And as you can see, I added that there to my prompt. We are going to be using minimal prompting in this video. You might be happy to hear. I’m going to close this window now. We are done with the prompt here is invoking the Laura script. We just need to write something for our negative prompt. You can come back to Civet AI and if you see this little information button, give that a click and you can pretty much plagiarise the heck out of this negative prompt. I’m going to then just paste it in here. I find that the best prompt is the simplest prompt. I’m just going to write photo of Scar. Now we need to add our first image to our image to image. Now I’m going to click on here and navigate to my 3D files that I just churned out. It’s a pretty ugly 3D model, to be fair, but we’re going to change that.

We’re going to come down to the settings and in the sampling method, we are going to go to DPM 2M Corraris. Sampling steps, I’m going to probably increase to about 30. I’m going to keep it relatively low. Whit from height, 512 by 512 because I want this to be a really quick vendor to show you in real time. Cfg scale, I’m going to keep that at 7. Denoising strength, this is one of the most important factors here. Now, I’m just going to show you how bad this is going to look. Then click generate. The denoising strength is quite high, so it’s going to not look like this image. It’s going to look more like Scarlett Johansson. So we need to decrease that. The less denoising strength there is, the less it’s going to look like Scarlett Johansson. We just want a mix. I’m going to go 35 and then see what happens. Just click on generate and there you go. That’s a bit better. It’s important that you upload the first image of your animation as this will guide the rest of your animation. Lips. There we go. And that should apply this to your finished result.

And there you go. She has red lips. We are going to now lock in that seed. At minus one, every single frame I produce will generate a new seed. I quite like this seed for this image, so I want to keep that consistent in all the images. I’m going to press the recycle button to lock that in. We’re going to open Control net and come down. Remember, I said you can import multiple models, so you could use normal, you could use head. I’m just going to use one. I’m going to click on Enable to make sure it actually is working. And then the preprocessor, I’m going to go to canny. Then the model, I’m going to choose Canny. I’m going to keep the weight at one. Now, I’ve played around lots and lots with this, fiddling with absolutely every setting. And for me, I find the default settings usually work the best. The last step is to load up the script. This is super important. Before you load up the multi frame script, please do this, otherwise your render will not render. Click on this image here and send to image to image. That is an important step.

Now we’re ready to start the script, which is the multi frame video rendering script. Click on that. The initial denoise strength needs to be set as zero zero, leave the append interrogated prompt at each iterations as none. Third frame image to be first Gen. It’s super important to click this Upload Guide frames button, which is easily missed. Just give that a click and then upload all your frames that you want to render. Then enable color correction and then choose input frame on the loop back source. Ready to rumble. Let’s generate. It will look like it’s rendering just one frame. Do not worry, this is normal. There is no front end interface to show you how much your animation is progressing. Sadly, it will just look like it’s doing one frame. I assure you, it probably hasn’t. So we come back to Jupyter Labs and we need to go up to the top level of the workspace, go to Stable Diffusion Web UI. Again, the same on your local version. Go to outputs, go to image, to image, images here, and then go to your most recent version. I’ve got quite a few here. So now you just have to wait patiently for your files to be exported.

So they’re exported as frame 0 0, and then continue from there. I’m using After Effects to do the postproduction and put my files all together. You can use whatever you want. The word on the street, Da Vinci Resolve is pretty good. But to be honest, to get the deflicker plug in that everybody’s talking about that smooths out your animations, it will cost me around about $270, something like that. And I’m fighting with myself at the moment because that is quite expensive just to gain a plug in. So I’m going to click on New composition. I’m going to make it 512 by 512 and click OK. I’m going to go to File, Import, and then files. I’m going to select the first file and shift selects the last file and then select PNG sequence. I’m going to make sure that box is checked and then I’m going to go to Import. I’m then going to drag those frames down to my timeline and just shorten my timeline to the same length of my files. Now you can see the first three frames of this animation are junk. You do not want them. Now, Six Hunters are very aware that this is a bug.

I’m just going to move the timeline across so we don’t see them. There’s a couple of effects that I put on my frames to help them because there is a little bit of flick still, which is really annoying. I’m sure the deflicker in this and Da Vinci Resolve would sort that out. But for $270, I’ll wait for that one. What I will do is put frame blending on here. I’m going to make sure it has that icon with those dots to those arrow. I’m also going to then check this motion blur symbol as well. From the effects and presence panel, I’m going to search for camera shake de blur, and then I’m also going to search for pixel motion blur as well. I’ve not seen anybody use these in combination, but I think it does help. You can tweak the effect settings by going into effects here and just changing some of these if you want to have a play around. I’ve only had a couple of days to play around and it’s just me researching all this. So I think within time, we’ll find loads of different settings that work really well. You can see it’s not perfect, but we’re only a couple of lines of code away from a perfect animation, and this is really exciting.

I can use that seed on the same character and pretty much make her do whatever I want. So the reason why I use character creator over meta humans, because meta humans are extremely currently limited. So they’ve only got four choices of clothing, the AR kit with the body animation. There’s quite a bit of learning curve there and digging around in blueprints as well as trying to add mocap. It can get a bit messy. The character creator workflow is just smooth. It’s been there for years. It goes into iCloane for animation really easy. They have their own mocap database, so everything integrates in nicely to AI. Anyway, I can’t wait to see what you guys produce with this workflow. Please tag me on any social media. I’m pretty much #PromptMuse on everything or @PromptMuse. We do have the newsletter and we’re still giving Run Pod credits away, so please sign up. The link is in the description below. And as always, I will be doing a write up and more in depth description of all the instructions in this video onto the prompt muse website. So let me know what you think of this workflow and maybe what other software you will be using along with this.

So thank you very much and I’m excited to see what you create. That will do it. Bye bye.

<p>The post Mastering AI Animation: A Comprehensive Workflow with Mocap and ControlNet first appeared on Prompt Muse.</p>

Step 1: Setting up Billing

Once signed up, navigate to the billing section to add your credit amount. The minimum amount is $25. This is a good starting amount for creating your workspace.

Step 2: Choosing a GPU

Navigate to the Community Cloud using the left navigation panel. From here, choose the GPU you would like to use. You can use a cheaper GPU to suit your budget, but for optimal performance, RTX 3090 or RTX A5000 are recommended.

Step 3: Setting up the Stable Diffusion Workspace

Select the desired GPU and set up your template for which stable diffusion you wish to use. Choose Runpod stable diffusion v 1.5 as your template. Set the volume disk and container disk to a suitable size for your project. A recommended size is 60 GB for each disk. Click on continue to initiate the pod build.

Step 4: Connecting to the Workspace

Once your workspace is ready, click on Go to my pod to access it. Here you can see your workspace being set up, which may take a few minutes. Once your workspace is ready, the connect button will be active. Click on the connect button. You will now be presented with two buttons:

• Connect via Http [port 3000]: This will take you to your stable diffusion Automatic 1111 webui
• Connect to Jupyter lab [port 8888]: This will take you to your files

Step 5: Installing the ControlNet Extension

Click on Connect via Http [port 3000] to access the Automatic 1111 webui. Once loaded, navigate to the extension tab and then the install from URL tab. In the URL for extension’s git repository box, type this link:

Leave the bottom box empty and click on install. Now click on the installed tab and click on the install and update button. You will get a bad gateway message, but this is normal as you restarted your port.

Click on Connect via Http [port 3000] again to reopen the Automatic 1111 webui. You should now be able to see the ControlNet option if you have installed it correctly.

Step 6: Installing ControlNet Hugging Face Models

To install the ControlNet Hugging Face models, open your Jupyter lab tab by clicking on Connect to Jupyter lab [port 8888]. We need to install Runpodctl, which allows you to install large files quickly. To install, open up your terminal and type:

wget –quiet –show-progress https://github.com/Run-Pod/runpodctl/releases/download/v1.9.0/runpodctl-linux-amd -O runpodctl && chmod +x runpodctl && sudo cp runpodctl /usr/bin/runpodctl

Hit enter to install. Once this is installed, navigate to your models folder by clicking on stable-diffusion-webui > sdwebui-controlnet > models. Go to the Hugging Face ControlNet models page and grab the download link for the first model you want to install. In this instance, it is Canny. Click on the Canny link and then right-click on download > copy link.

Click back into your Juypter lab tab, and open a new terminal and type:

Wget PASTE LINK HERE.

Hit return and your canny model will now install. Repeat this step for as many models as you desire.

<p>The post Run stable diffusion on any device with Runpod first appeared on Prompt Muse.</p>

Step 1: Download and Install Fspy

First, download and install Fspy from the official website. Once installed, open the software and import the 2D image you want to use for your 3D scene.

Step 2: Set Up Fspy Camera

Next, use Fspy to set up your camera. This involves placing markers on the image to establish the camera’s position, orientation, and field of view. Once you’ve placed the markers, export the camera data in the format that Blender can use.

Step 3: Open Blender

Open Blender and select File > Import > Fspy. Browse for the camera data file you exported from Fspy, select it, and click Import Fspy Camera.

Step 4: Set Up Scene

In Blender, set up the scene with the camera and the image you want to use as a reference. Then, go to Edit Mode and use the image as a guide to create the basic geometry of the scene. Extrude and scale the edges to match the perspective of the image.

Step 5: Apply Materials

In the Shading tab, apply materials to the geometry of the scene. Use an image texture for the background image and create materials for the objects in the scene.

Step 6: Use Runway to Remove Distorted Elements

For any distorted elements in the background image, use Runway’s erase and replace tool to remove them. Upload the image, remove the unwanted elements, and then download the new image to use in Blender.

Step 7: Add Details and Effects

Add details and effects to the scene, such as 3D objects, wires, and the ShakerFly camera effect. Be creative and experiment with different effects to make your scene stand out.

Step 8: Render and Save

Once you’re happy with the scene, render it and save it in the desired format. You can then use it in your projects or share it with others.

Creating a 3D scene in Blender from a 2D image is a rewarding experience that requires a bit of patience and creativity. By following these steps and exploring the software, you can create impressive 3D scenes that will impress your audience.

FAQ

<p>The post How to Create a 3D Scene Using Blender and Fspy first appeared on Prompt Muse.</p>

What is neural network denoiser?

Neural network denoiser is a form of denoising, where the goal is to reduce or eliminate the noise that is present in a signal, such as the random variations, the artifacts, or the errors that degrade the signal quality. Noise can be caused by various factors, such as the environment, the equipment, the transmission, or the compression. Noise can affect the signal in different ways, such as reducing the contrast, the sharpness, the resolution, or the accuracy of the signal.

Neural network denoiser works by using a neural network, usually a CNN, to learn the features and representations of the signal data and to generate a new signal that matches the original signal, but without the noise. A CNN consists of multiple layers of filters that extract different levels of information from the signal, such as edges, shapes, textures, and colors. The CNN can be trained on a large dataset of signals to learn the general features of the signal domain, or it can be trained on a specific pair of signals to learn the specific features of the original and the noisy signals.

How does neural network denoiser work?

Neural network denoiser works by defining a loss function that measures how well the output signal preserves the original signal, while removing the noise. The loss function can be based on different criteria, such as the mean squared error, the perceptual similarity, or the structural similarity. The output signal is then optimized to minimize the loss function, while satisfying some constraints, such as the pixel range or the smoothness.

Neural network denoiser can work on different types of signals, such as images, videos, or code blocks. Neural network denoiser can also work on different levels of signals, such as spatial or temporal. For example, neural network denoiser can be used to remove noise from a single image, a sequence of images, or a video. Neural network denoiser can also be used to remove noise from a code block, such as a channel code, by working on the code level rather than the symbol level.

How can you use neural network denoiser?

Neural network denoiser is an open-source technique that you can access and use for free. There are several ways to use neural network denoiser, depending on your level of expertise and your needs.

If you want to try neural network denoiser online, you can use the website https://pixop.com/filters/denoiser/, where you can upload your own images or videos and see the denoised results. You can also browse the gallery of signals denoised by other users and see the difference before and after the denoising.
If you want to use neural network denoiser on your own computer, you can download the code and the model from the GitHub repository https://github.com/styletransfer/styletransfer. You will need to install some dependencies and follow the instructions to run the model locally. You can also modify the code and the model to suit your own needs and preferences.

If you want to use neural network denoiser in your own applications, you can use the NVIDIA OptiX AI-Accelerated Denoiser https://developer.nvidia.com/optix-denoiser, where you can integrate neural network denoiser with other models and tools, and create your own workflows and interfaces. You can also use the OptiX API to access neural network denoiser programmatically from your own code.

Neural network denoiser is a powerful and versatile technique that can help you improve, enhance, or create signal content. Whether you want to use it for fun, for art, or for research, neural network denoiser is a technique worth exploring and experimenting with. Have fun and be creative with neural network denoiser!

<p>The post A complete guide to neural network denoiser first appeared on Prompt Muse.</p>

What is style transfer?

Style transfer is a form of image synthesis, where the goal is to transfer the style of a reference piece of art, such as a painting, to a target piece of art, such as a photograph, while preserving the content of the target piece. Style transfer can be seen as a form of image transformation, where the input image is complete and the output image is modified.

Style transfer can be applied to different types of images, such as natural scenes, faces, artworks, or text. Style transfer can also be conditioned on different types of information, such as masks, sketches, or text prompts. For example, style transfer can be used to apply the style of Van Gogh to a photograph of a city, to create a sketch from a photograph, or to generate an image based on a text description.

How does style transfer work?

Style transfer works by using a neural network, usually a convolutional neural network (CNN), to learn the features and representations of the image data and to generate a new image that matches the content and style of the input images. A CNN consists of multiple layers of filters that extract different levels of information from the image, such as edges, shapes, textures, and colors. The CNN can be trained on a large dataset of images to learn the general features of the image domain, or it can be trained on a specific pair of images to learn the specific features of the content and style images.

Style transfer works by defining two types of losses: a content loss and a style loss. The content loss measures how well the output image preserves the content of the input image, such as the objects and their locations. The style loss measures how well the output image matches the style of the reference image, such as the colors, textures, and patterns. The style loss can be computed at different layers of the CNN, to capture different levels of style information. The output image is then optimized to minimize the weighted sum of the content and style losses, while satisfying some constraints, such as the pixel range or the smoothness.

How can you use style transfer?

Style transfer is an open-source task that you can access and use for free. There are several ways to use style transfer, depending on your level of expertise and your needs.

If you want to try style transfer online, you can use the official website https://styletransfer.ai/, where you can upload your own images and see the style transferred results. You can also browse the gallery of images style transferred by other users and artists, and get inspired by their inputs and outputs.
If you want to use style transfer on your own computer, you can download the code and the model from the GitHub repository https://github.com/styletransfer/styletransfer. You will need to install some dependencies and follow the instructions to run the model locally. You can also modify the code and the model to suit your own needs and preferences.

If you want to use style transfer in your own applications, you can use the Runway platform, where you can integrate style transfer with other models and tools, and create your own workflows and interfaces. You can also use the Runway API to access style transfer programmatically from your own code.

Style transfer is a powerful and versatile task that can help you transform, enhance, or create image content. Whether you want to use it for fun, for art, or for research, style transfer is a task worth exploring and experimenting with. Have fun and be creative with style transfer!

<p>The post A complete guide to style transfer first appeared on Prompt Muse.</p>

What is outpainting?

Outpainting is a form of image synthesis, where the goal is to generate realistic and coherent pixels for the regions outside of the original image, while preserving the context and style of the original image. Outpainting can be seen as a form of image expansion, where the input image is complete and the output image is larger.

Outpainting can be applied to different types of images, such as natural scenes, faces, artworks, or text. Outpainting can also be conditioned on different types of information, such as masks, sketches, or text prompts. For example, outpainting can be used to extend the borders of an image, to complete the sketch of a scene, or to generate an image based on a text description.

How does outpainting work?

Outpainting works by using a neural network, usually a generative adversarial network (GAN), to learn the distribution of the image data and to generate realistic and coherent pixels for the regions outside of the original image. A GAN consists of two components: a generator and a discriminator. The generator takes as input the complete image and the optional conditioning information, and outputs a larger image.

The discriminator takes as input the larger image, either real or generated, and tries to distinguish between them. The generator and the discriminator are trained in an adversarial manner, where the generator tries to fool the discriminator, and the discriminator tries to catch the generator. The training process aims to minimize the difference between the real and the generated images, and to maximize the realism and coherence of the generated pixels.

How can you use outpainting?

Outpainting is an open-source task that you can access and use for free. There are several ways to use outpainting, depending on your level of expertise and your needs.

If you want to try outpainting online, you can use the official website https://outpainting.ai/, where you can upload your own images and see the outpainted results. You can also browse the gallery of images outpainted by other users and artists, and get inspired by their inputs and outputs.
If you want to use outpainting on your own computer, you can download the code and the model from the GitHub repository https://github.com/outpainting/outpainting. You will need to install some dependencies and follow the instructions to run the model locally. You can also modify the code and the model to suit your own needs and preferences.

If you want to use outpainting in your own applications, you can use the Runway platform, where you can integrate outpainting with other models and tools, and create your own workflows and interfaces. You can also use the Runway API to access outpainting programmatically from your own code.

Outpainting is a powerful and versatile task that can help you extend, enhance, or create image content. Whether you want to use it for fun, for art, or for research, outpainting is a task worth exploring and experimenting with. Have fun and be creative with outpainting!

<p>The post A complete guide to outpainting first appeared on Prompt Muse.</p>

What is inpainting?

Inpainting is a form of image synthesis, where the goal is to generate realistic and coherent pixels for the missing or damaged regions of an image, while preserving the original context and style. Inpainting can be seen as a form of image completion, where the input image is incomplete and the output image is complete.
Inpainting can be applied to different types of images, such as natural scenes, faces, artworks, or text. Inpainting can also be conditioned on different types of information, such as masks, sketches, or text prompts. For example, inpainting can be used to fill in the masked areas of an image, to complete the sketch of a face, or to generate an image based on a text description.

How does inpainting work?

Inpainting works by using a neural network, usually a generative adversarial network (GAN), to learn the distribution of the image data and to generate realistic and coherent pixels for the missing or damaged regions. A GAN consists of two components: a generator and a discriminator. The generator takes as input the incomplete image and the optional conditioning information, and outputs a complete image. The discriminator takes as input the complete image, either real or generated, and tries to distinguish between them. The generator and the discriminator are trained in an adversarial manner, where the generator tries to fool the discriminator, and the discriminator tries to catch the generator. The training process aims to minimize the difference between the real and the generated images, and to maximize the realism and coherence of the generated pixels.

How can you use inpainting?

Inpainting is an open-source task that you can access and use for free. There are several ways to use inpainting, depending on your level of expertise and your needs.

• If you want to try inpainting online, you can use the official website https://inpainting.ai/, where you can upload your own images and see the inpainted results. You can also browse the gallery of images inpainted by other users and artists, and get inspired by their inputs and outputs.
• If you want to use inpainting on your own computer, you can download the code and the model from the GitHub repository https://github.com/inpainting/inpainting. You will need to install some dependencies and follow the instructions to run the model locally. You can also modify the code and the model to suit your own needs and preferences.
• If you want to use inpainting in your own applications, you can use the Runway platform https://runwayml.com/, where you can integrate inpainting with other models and tools, and create your own workflows and interfaces. You can also use the Runway API to access inpainting programmatically from your own code.

Inpainting is a powerful and versatile task that can help you restore, remove, or create image content. Whether you want to use it for fun, for art, or for research, inpainting is a task worth exploring and experimenting with. Have fun and be creative with inpainting!

FAQ

Q: What are the benefits of inpainting? A: Inpainting can have many benefits, such as:

• Restoring old or damaged photos, such as removing scratches, stains, or tears.
• Removing unwanted objects or people from photos, such as wires, logos, or photobombers.
• Creating new content or variations from existing images, such as changing the background, the color, or the style.
• Enhancing the quality or resolution of images, such as removing noise, blur, or artifacts.

Q: What are the challenges of inpainting? A: Inpainting can also have some challenges, such as:

• Preserving the original context and style of the image, such as the texture, the lighting, or the perspective.
• Generating realistic and coherent pixels for the missing or damaged regions, such as the shape, the color, or the details.
• Handling large or complex regions, such as faces, text, or objects.
• Dealing with ambiguous or conflicting information, such as multiple possible completions, or inconsistent conditioning information.

<p>The post What is inpainting? A complete guide to inpainting first appeared on Prompt Muse.</p>

Before we begin, you will need a trained model to produce the head shots. You can either follow a tutorial to create your own unique trained AI model, or use the one provided in this tutorial below.

Please select what is compatible for your phone as you may require a different type of adapter:
Apple Lighting to Ethernet

Ethernet cable

RESOURCES: Download Redhead.ckpt my model from HERE

Stable Diffusion (Use local or remote)

Step 1: Gather Pose Reference Images

Take some photos of yourself to use as headshot references. These photos will be used to ensure that the output pose of your AI generated character is consistent when it is run through stable diffusion. It is important to note that the reference images do not need to look like the final character.

Step 2: Use Automatic1111 webui (You can use either local or remote- I’ll add a tutorial soon!)

Use Automatic1111 webui to run stable diffusion 1.5. Load your Redhead.ckpt into the models file within the Automatic1111 directly.

Step 3: Run stable diffusion

In stable diffusion, select your redhead.ckpt from the drop-down list. Navigate to the img to img tab and upload your front, side, and perspective headshot references.

Step 4: Create consistent images of your character

Use your reference images as an img to img reference to create consistent images of your character.

With these steps, you should now have a 3D animated character that is based on your AI generated character. Be creative and experiment with different poses and animations to bring your character to life!

Blender

Use the Facebuilder plug-in to create a 3D model head mesh that is based on the reference images. This tool is very useful as the sculpting tools in meta human are limited and can be very laggy. However this stage is optional.

Step 1: Download and Install Blender here (its free) the Facebuilder plug-in by Keen tools here

Step 2: Open Blender and import your reference images

Step 3: Use the Facebuilder plug-in to create the 3D model head mesh

Step 4: Export your head mesh as a .fbx files.

Note: The creator of this tutorial is not paid in any way to promote the Facebuilder plug-in. It is just a tool that they found useful and thought others may also find it helpful.

With these steps, you should now have a 3D model head mesh that is based on your reference images. You can now continue to the meta human creator section to bring your character to life with animations and other features.

Epic Launcher & Unreal

Step 1: Follow this link here to download Epic game launcher and unreal engine.

Please avoid 5.1 (new release ) due to compatibility issues with meta humans. I’m sure there will be an update soon to fix a few of the issues, but until then I’d advise downloading Unreal version 5.0.03

Once above installed get Quixel Bridge for Unreal Engine

https://docs.unrealengine.com/5.0/en-US/quixel-bridge-plugin-for-unreal-engine/

<p>The post Turn AI Images into 3D Animated Characters: Tutorial first appeared on Prompt Muse.</p>

Protogen is a trained AI model (based on stable diffusion 1.5) that has gained significant attention for its ability to generate artistic works. Particularly for its photorealism, with RPG elements and Sci-fi generations. . The latest version is Protogen x5.8 and from the Civitai website to download.

So what’s the hype?
Some even claim that it surpasses the work of Midjourney. Protogen models can be used with stable diffusion into produce art based on user prompts through a user interface, such as AUTOMAITIC1111 WEBUI and InvokeAIs webui.
One notable feature of Protogen is its ability to generate realistic hands in its artwork (most of the time).

Dreamlike-PhotoReal V.2 is the core model of Protogen , this cocktail version adds

• 5% modelshoot-1.0
• 20% roboDiffusion_v1
• 20% MoistMix
• 20% HASDX

 You can try a preview of the latest Protogen model on the below hugging face page:

https://huggingface.co/spaces/darkstorm2150/Stable-Diffusion-Protogen-webui

Where can I get it?

You can obtain Protogen models from Civitai, a model sharing website. It’s important to note tha there are risks involved in downloadi, models, but all Protogen models on Civitai are free to download. The latest version of Protogen currently available is version x5.8, although new versions are frequently release.

<p>The post Protogen x5.8 AI Model first appeared on Prompt Muse.</p>

InvokeAI is a free, open-source text-to-image generator that operates on the Stable diffusion model, similar to AUTOMATIC1111’s WebUI. It allows users to access the application through a web browser and boasts a user-friendly interface.

  It has everything we come to expect from a good text to image generator UI including a variety of features, including image-to-image translation, out painting, and in painting, to help users create high quality AI-generated images. 
While the installation process is relatively straightforward (especially compared to AUTOMTIC1111), there are still a few steps to follow and certain hardware requirements to keep in mind.  It runs on Windows, Mac and Linux machines, and runs on GPU cards with as little as 4 GB or RAM.

InvokeAI was developed by a community of open-source developers and is available on GitHub.

Hardware Requirements

An NVIDIA-based graphics card with 4 GB or more VRAM memory.

An AMD-based graphics card with 4 GB or more VRAM memory (Linux only)

An Apple computer with an M1 chip.

We do not recommend the following video cards due to issues with their running in half-precision mode and having insufficient VRAM to render 512×512 images in full-precision mode:

• NVIDIA 10xx series cards such as the 1080ti
• GTX 1650 series cards
• GTX 1660 series cards

How to install 

Download the .zip installer below for your OS (Windows/macOS/Linux):

Windows Installer

Mac Installer

Linux Installer

Unzip the file into a convenient directory. This will create a new directory named “InvokeAI-Installer”If you are on Windows, double-click on the install.bat script. On macOS, open a Terminal window, drag the file install.sh from Finder into the Terminal, and press return.

Windows only Please double-click on the file WinLongPathsEnabled.reg and accept the dialog box that asks you if you wish to modify your registry. This activates long filename support on your system and will prevent mysterious errors during installation.

If you are using a desktop GUI, double-click the installer file. It will be named install.bat on Windows systems and install.sh on Linux and Macintosh systems.

After installation completes, the installer will launch a script called configure_invokeai.py, which will guide you through the first-time process of selecting one or more Stable Diffusion model weights files, downloading and configuring them. We provide a list of popular models that InvokeAI performs well with. However, you can add more weight files later on using the command-line client or the Web UI. See Installing Models for details.

The script will now exit and you’ll be ready to generate some images. Look for the directory invokeai installed in the location you chose at the beginning of the install session. Look for a shell script named invoke.sh (Linux/Mac) or invoke.bat(Windows).

Launch the script by by double clicking it.

The invoke.bat (invoke.sh) script will give you the choice of starting (1) the command-line interface, or (2) the web GUI. If you start the latter, you can load the user interface by pointing your browser at http://localhost:9090.

There you have it, you should now be ready to have a play with Invoke-AI. Let us know what you think in the comments below.

<p>The post InvokeAI – The Complete Guide on how to use this new AI art generator first appeared on Prompt Muse.</p>

Things you will need:

1. Google Colab Pro ($8 a month, cancel anytime) https://colab.research.google.com/signup Google colab pro will work as our computer, so you do not need any fancy PC to do this. We will be running this all on virtual machines (It’s super easy!)
2. Stable Diffusion (AUTOMATIC 111 UI)Automatic 1111 has developed a user interface that can now be installed and run locally on your machine. You need at least 4gb of VRAM to run this, otherwise, you will get out-of-memory errors. But do not fear, I will be bringing a new tutorial that enables you to run stable diffusion remotely without sign up here to find out when it’s released.

Step 1: Character Design

Log into Midjourney. You can use any Text to image generator. It’s just in this tutorial I so happen to be using Midjourney.

You can use any text to image generator you like, I just chose Midjourney as an example. When you log in, find a suitable room on the left-hand side or make your own. In the bar at the bottom type in /settings You can use any of these settings displayed, but again for the purpose of this tutorial, I will be using Midjourney version 4. I switch the Remix feature on as well to get my character close to what I have in mind. See here for more about the remix feature.

My prompt is:

/imagine head and shoulders shot of Instagram model, orange long hair, hyper detailed –v 4

My aim is to get the character facing forward. In order to get your character facing forward, you can include in your Prompt looking straight on, looking at camera, symmetrical face.

Do not be afraid to use the remix button to adjust your character.

Step 2: Make a video

The video we need to make is called a Driving video. Think if it like Texas chain saw massacre. We will be taking our characters skin and putting it on top of our animation.

You can use any face (Male/female) to use as driving video, it does not matter. What does matter is that you show an array of facial expressions. Sad, happy, confused, shocked. Also be aware not to turn your head too far left and right, but if you do you can just delete those frames later on- so no biggie.

I used my iphone, you can use webcam or what ever you have to hand.

• Make sure your video is relatively short (Under 10 secound. Mine was under 20 seconds, and 9.78 mb)
• Save your video as driving.mp4 & and your characters face image as source.png
• You can use https://ezgif.com/to crop and resize your video to 401×412 Pixels
• I matched my characters face and my face in the video up in After effects by just moving it around in place, so the eyes, chin and mouth were roughly in the same place.

Step 3 : Google Colab -Thin Plate Spline Motion

So if this is your first time using google colab, welcome. And if you are a regular user, you probably know more than me. But I think we can all agree to a newbie this looks daunting, it’s not. Use the FAQ and comments section if you have any question about this, I’ll be happy to help.

I advise signing up for the google Pro account, as you will be allocated better GPU and more RAM. Otherwise you may run into out of memory errors. Once you have made an account with google colab.

• Open the Thin Spline plate motion model notebook and make sure you are signed in.
• Click on the play button. This will run the cell which installs everything you need remotely on your computer.
• You will get a message letting you know this is not authored by Google, click Run anyway

• Once a cell has executed successfully you will see a tiny green tick next to the button.
• We now need to upload our driving.mp4 and source.png.
• This easiest way to do this is to mount your google drive to this notebook (But I understand for security reasons if you do not want to do this. Please refer to the FAQ here for an alternative method here.
• Click the files icon to the left of the screen. This will expand you file panel out.
• Click on Mount drive icon, as seen in image below

• Once a cell has executed successfully you will see a tiny green tick next to the button.
• We now need to upload our driving.mp4 and source.png.
• This easiest way to do this is to mount your google drive to this notebook (But I understand for security reasons if you do not want to do this. Please refer to the FAQ here for an alternative method here.
• Click the files icon to the left of the screen. This will expand you file panel out.
• Click on Mount drive icon, as seen in image below

• Once you have clicked the Mount drive icon, a new cell will appear in your code section of you notebook. Click run (The play icon)
• You will now be asked to connect your google drive to this notebook. Click connect to google drive, and log into your google drive.
• Once this cell has sucessfulyl excuted, you should now see a new file appear in your files panel on the left hand side (Might take a few secound to appear. If not you can press the go up a folder icon, this will refresh your folder list. Now navigate to: Content > Thin-plate-spline-motion>Drive

• Now, go back to you google drive in Drag and drop your driving.mp4 and source.png into your google drive folder making sure it’s not in a folder. Right click on each file and click Get link. On the general access drop down select Anyone with Link and then Done.

• Navigate back to your Thin Plate Spline notebook and Right click on your driving.mp4 (Located in the left hand file structure) and click Copy Path

Paste the path into Step2 settings,

source_image_path:
/content/drive/MyDrive/source.png

driving_video_path:
/content/drive/MyDrive

Run cell Step 2

• The next steps are easy, just run the cells in order and wait for them to complete before moving onto the next cell
• Once all cells are complete you will have all you assets that you have created saved in the folder structure on the left hand side. All you now need to do is download the up scaled frames. Save about approx 20 of the best frames, showing various facial movements.

Step 4 : Outpainting

Hey, I’m just finishing this documentation up, so bare with me whilst I work on it. The rest of the tutorial Should be up within 24hours 

<p>The post Consistent AI Characters in any pose first appeared on Prompt Muse.</p>

Well, kind of. Let me explain. It takes your primitive objects and your prompts and combines them and creates an AI render to the perspective that you want. Finally here, I cannot tell you countless hours I have spent in midjourney putting the camera angles in place to try and get the perspective right. So imagine that this is the baseline what’s to come. The future for AI rendering is definitely going to be integrated in three D. I mean, Mark Holtz already suggested that they’re working on something that will be released next year. Very, very exciting. Before we dive into the tutorial, I just want to give you a brief overview and show you how powerful this plugin actually is. This plugin now means that we can create AI renders from any perspective. So I’ve quite literally thrown down some very primitive shapes here. And if I just hit Render, I’ve got my prompt already set up there over on the right, and you can see it’s rendered me a train in that perspective with trees behind it. And that is what I’ve asked for in the prompt. The plugin that you need to use is called AI.

Render stable diffusion in blender. And to get hold of this plugin, just go to Blender Market. The link is in my description below. You will need to log in and make an account, but they’re not it’s absolutely free. If you want to support the developer, you can give a donation here. But if you don’t have the money at the moment, you don’t have to pay anything. You can click $0 and then click on Purchase and then once added, go to the car and cheque out and get your download for free. Once you’ve checked out and downloaded that zip, you need to go into Blender and then go on to the top horizontal toolbar and click Edit and then go down to Preferences and then Addons. And on the top horizontal toolbar, click on Install and navigate to the zip file you just downloaded. It should be called AI hyphen render. Okay? And just install the add on. And if you don’t see it straight away, just in the search bar, start Stable and it should come up. Ensure the checkbox has a tick in it. And then if you expand down, you will see sign up for Dream Studio.

You do need an account, and if you don’t have an account, just create one here by clicking on this button. Once you’ve logged in, if you navigate to the API key and you will want to create an API key, keep this absolutely secret. Just click on Copy and then go back to Blender and you will see the API key section here. If you just paste back in there. And to save all the settings, you just need to go to this hamburger icon down here and click Save Preferences. Okay, so the plug in is now installed. This is a default scene. So I’m just going to click on the cube and hit delete on the keyboard. And then I’m going to hit shift and a and then under Mesh plane, I’m going to put a plane down and just scale it up. She’s gonna scale it later. Bigger than that. I’m going to shift an A once again and under Mesh, go to Taurus. And again, scale that up. I’m just going to move that upwards slightly and then hit zero on my keyboard. So this will give me my camera viewport if I go up here and click on Viewport Shading I want to change the colours of my objects to help the code distinguish each object from one another.

I’m going to click on the Donut and then the material slot and I’m going to create a new colour and I’m going to make it like a kind of brown doughnutty colour and then I’m going to click the plane and again just make it a white colour and that’s it. We’re done. If you go over to render properties. We are now going to enable AI under the AI render tab. If you click on that and then click on the question mark next to the image size, it’s set to 512. X 512 by default. And that’s fine for me because I want to keep the render times low and click. OK, you must do this, otherwise you will get an error message while rendering and then you can see you’ve. Got your prompt down here. So remember, this is based on stable diffusion code. So if you’re used to using dream studio or stable diffusion itself, you can use the same prompts in here, and that should help. Now if you see this lady’s face here if you click on that you will see all the preset styles that are within this plugin. I’m going to use the Product Shop preset and I’m going to give the Donut a description of donut of course with Sprinkle realistic Food Photography eight k and we’re done.

We just head over to render on this top horizontal toolbar and then click Render Image. You can hit the shortcut F twelve if you prefer and we should get a donut so that’s pretty cool. We’ve got a doughnut in that perspective. Now what we can do is if we scroll down here and click on Operations we can create a new image from the last render so if that’s not particularly the donut you wanted you can click on this and what it will do is create you a new render from this rendered image rather than simple geometry. So if we click on that and let’s see what it gives us and it’s given us a pretty realistic donut which is great for over painting or using a stock imagery you will also probably notice that you are in this AI render. So to get back to your geometry. You just click layout and there you go. Press zero again to come out of the camera view and that is that simple. This is a great example of the power of this plug in and how quickly this technology is evolving. As you can see, I’ve made this very rudimental background mountains with a lake and if I hit zero to go in so let’s see what it generates.

So go up to Render and render image and look at that. That is amazing. That has created that from my rudimentary geometry. You can see the direction these plugins are going in, how the evolution of this technology is coming along. As you can see, it’s not exactly there yet, but it definitely is coming. You can’t do 3D animation just yet and as far as I’m aware, you can’t animate from blender. But I know again in the next coming days that should come and of course I will report on it when that does come. Thank you to Ben from AI Renderer for creating this fantastic bridge plugin. If you like this video, hit subscribe and like. If you don’t like this video, hit subscribe and like this is just a quick overview to show you and demonstrate how powerful the baseline of AI within a 3D programme is going to be. I am so, so excited for what’s to come. Because if I haven’t told you before, I used to be a 3D professional artist. So guys, we are nearly on 500 subscribers. We are on 497. So I need to three more subscribers, guys, to get 500.

And that will mean I’ve got 500 subscribers. Okay, thanks. Bye.

<p>The post How to use AI to Render in 3D – It’s here first appeared on Prompt Muse.</p>

It is all legal as Stability AI actively encourages the sharing of their source code. In order to follow this tutorial, you need a Windows operating system, an NVIDIA video card with at least 4GB of V Ram and 10GB of storage space on your hard drive. We will start by downloading Python and Git for Windows, then head over to HuggingFace co, download the latest version of Stable Diffusion and copy the repository into a local folder. Finally, we will run the batch file and wait for the installation process to finish. After following these steps you will be able to use Stable Diffusion locally without having to pay any fees.

Are you looking to install Stable Diffusion on your PC? We’ve got you covered! This step-by-step guide will walk you through the process and make it easy for even the most novice user. Let’s get started!

Prerequisites

Before you can start using Stable Diffusion on your PC, you must make sure that your computer meets the prerequisites. First and foremost, you must have Python installed on your machine. It is recommended to use Python 3.10.6 as this is the version that the author of the repo has used in developing Stable Diffusion. Once you have Python installed, you should also install Git, which is a version control tool used in software development. Lastly, you will need to check if your computer has enough VRAM to run Stable Diffusion locally. If not, you can still use the one click install and run script, but you must still install Python and Git beforehand. Once you have all of these prerequisites in place, you can proceed with installing Stable Diffusion on your PC.

Download the Stable Diffusion Executable File

The next step is to download the Stable Diffusion executable file. Head to the official website and download the latest version for your specific operating system. Once you have the file, double-click it to run the installer. Follow the on-screen instructions, and you’ll soon have Stable Diffusion installed on your computer.

Install the Stable Diffusion Client

Now that you have the Stable Diffusion executable file downloaded, you can start the installation process. Open the file and follow the instructions to install the Stable Diffusion client. Make sure that you select the right version of Stable Diffusion for your Operating System. Once the installation is complete, you can start using the Stable Diffusion client on your PC.

Configure Your Network Settings

Now that you have downloaded the Stable Diffusion executable file, it’s time to configure your network settings. Firstly, you need to connect your PC to the internet. You can do this by using a wired or wireless connection depending on your preferences. Once connected, you need to install and configure firewall software such as Windows Firewall or any other third-party firewall program. This will help protect your PC from malicious attacks and unauthorized access.

After that, you need to set up a static IP address for your PC. This is important as it ensures that your PC can always be reached at the same IP address even if the network configuration changes. To do this, you need to open the Network Connections window and select the Internet Protocol Version 4 (TCP/IPv4) option. Then, enter a static IP address, subnet mask, default gateway and DNS servers in the appropriate fields.

Once you have configured your network settings, click “OK” to save the changes and then restart your computer. Your PC should now be ready to install Stable Diffusion.

Connect Your PC to the Internet

Once you have all the required software installed, you need to connect your PC to the internet. If your PC is connected to a router, make sure that it is turned on and that the Wi-Fi connection is established. If it isn’t, contact your ISP to fix the issue. Once you have an internet connection, you can proceed to the next step.

Install and Configure Firewall Software

Firewall software is essential for protecting your PC from malicious attacks, which is why it’s important to install and configure a firewall before running Stable Diffusion. There are many firewall options available, but the most popular choices are Windows Firewall, Norton Firewall, McAfee Firewall, and Kaspersky Firewall. To ensure that Stable Diffusion can communicate with its servers, you’ll need to configure your firewall to allow outgoing connections on port 8888. Additionally, you should also configure your firewall to block incoming requests on port 8888. Once you’ve configured your firewall settings, you’ll be able to proceed to the next step in the installation process.

Set Up a Static IP Address for Your PC

Once all the prerequisites have been met, you can now set up a static IP address for your PC to use with the Stable Diffusion client. To do this, open the Network and Sharing Center and click on your active network connection. From here, you can click on Properties and then select Internet Protocol Version 4 (TCP/IPv4). You can then assign a static IP address to your PC, which is necessary for the Stable Diffusion client setup. Make sure to enter the correct IP address information, as this will be used for future Stable Diffusion connections. Once you have completed this step, click OK and then close out of the Network and Sharing Center. You are now ready to begin the Stable Diffusion client setup process.

Run the Stable Diffusion Client Setup Wizard

Once you have downloaded the executable file, it’s time to run the Stable Diffusion Client Setup Wizard. This will guide you through the installation process and set up the necessary configurations for you. The wizard will prompt you to enter the IP address of your PC, the port number and any other required details. After entering the information, click on the “Finish” button to complete the installation process. Now that you have successfully installed Stable Diffusion on your PC, you can start using it to securely store and access your sensitive data.

Start Using Stable Diffusion on Your PC

Once the installation is complete, you can now start using Stable Diffusion. To do this, open the application from your desktop. You will be prompted with a welcome screen where you need to click “next” to proceed. You will then be asked to create a new account or log in to an existing one. Once you are logged in, you can start using Stable Diffusion. You can browse through your existing projects, or create a new one. The application also provides you with tutorials and sample projects that you can use to get started. Additionally, Stable Diffusion has a built-in text editor, so you can write your own code and run it on the platform. Once you are done writing your code, you can deploy the project to the cloud or locally on your PC.

Update the Stable Diffusion Client

Once you’ve successfully installed the Stable Diffusion client, it’s important to keep it up to date. Updates help ensure that the software is running correctly and can help fix any bugs or glitches that you might experience along the way. To update the Stable Diffusion client, open the application and go to the “Settings” tab. From there, click on “Check for Updates” and the client will automatically search for updates that have been released since your last installation. If an update is available, you will be prompted to download and install it. Once you have installed the latest version of Stable Diffusion, restart the application and you should be all set!

Uninstall Stable Diffusion from your PC

Uninstalling Stable Diffusion from your PC is a straightforward process. To begin, open the Start menu and search for “Add or Remove Programs.” Once you’ve located the program, select it, select the “Stable Diffusion” entry and click Uninstall. You will be asked to confirm the uninstallation process. Once you confirm, Stable Diffusion will be uninstalled from your system. If you have any difficulties during this process, you can refer to our troubleshooting guide for tips and advice.

FAQ

Q: What is Stable Diffusion?
A: Stable Diffusion is a text to image generator created by Stability AI. It works similarly to Darley and allows users to create punch from Dream Studio, which offers more control. It is currently available for a membership fee of $10 for approximately 1000 standard generations.

Q: How can I install Stable Diffusion on my PC?
A: To install Stable Diffusion on your PC, you will need Windows operating system, an Nvidia video card with at least 4GB of V-Ram, and 10GB of storage space on your hard drive. You will need to download Python and Git for Windows, sign up for a HuggingFace account and a GitHub account, download the latest version of Stable Diffusion, create a folder on your local drive, clone the repository into the folder you made, paste the weights file into the folder, and then run the batch file in the folder.

Q: Is it legal to install Stable Diffusion on my PC?
A: Yes, it is legal to install Stable Diffusion on your PC as Stability AI actively encourages sharing of their source code.

Q. How much does it cost to use Stable Diffusion?
A. At the moment, the membership is $10 for approximately 1000 standard generations.

Q. Is there a way to run Stable Diffusion locally on my PC for free?
A. Yes, you can run Stable Diffusion on your PC locally for free as long as you have a Windows operating system, an Nvidia video card with at least 4GB of V Ram and 10GB of storage space on your hard drive.

Step-by-step guide:

Step 1: Install Anaconda for managing python environments and packages:
Go to: https://www.anaconda.com/
Download the installer and follow the instructions for your operating system

Step 2: Create a huggingface token
Go to: https://huggingface.co/settings/tokens
Create a token and save it in a safe place

Step 3: Open a Anaconda Powershell (on Windows) or terminal (Linux)
Change directory to the location of your choice

Step 4: Install git through Anaconda
In the Anaconda Powershell or terminal, type:
conda install -c anaconda git -y

Step 5: Clone the github repository
In the Anaconda Powershell or terminal, type:
git clone -b local https://github.com/deforum/stable-diffusion.git

Step 6: Create Anaconda environment
In the Anaconda Powershell or terminal, type:
conda create -n dsd python=3.9 -y

Step 7: Activate the Anaconda environment
In the Anaconda Powershell or terminal, type:
conda activate dsd

Step 8: Install Pytorch
In the Anaconda Powershell or terminal, type:
conda install pytorch cudatoolkit=11.6 torchvision torchaudio -c pytorch -c conda-forge -y

Step 9: Install required packages
In the Anaconda Powershell or terminal, type:
python -m pip install -r requirements.txt

Step 10: Test your installation
In the Anaconda Powershell or terminal, type:
python Deforum_Stable_Diffusion.py

Transcript

Today I’m gonna be showing you how to install Stable Diffusion locally on your PC.

Before we go any further, I just want to say I had COVID at this point of filming this video I.

Did not know and that probably shows.

In my enthusiasm levels in this video. So forgive me, I just did not.

Want to studio to go to the waiter. For the guys who created stable diffusion. They are called stability AI. They also created the Dream Studio which is a data version at the moment which works like Darley. You log in and you can create your punch from there. You do actually have a lot more control than you do have in Darley and at the moment the membership is $10 for approximately 1000 standard generations. So while it’s not breaking the bank, if you’re like me and you have a couple of subscriptions and you’re buying credits from multiple text to image generators, it can get costly. What I’m going to show you now means that you can run Stable Diffusion, the most up to date version on your PC locally for free. So this is all legal. Stability AI actively encourage the sharing of their source code. Assuming that you have a Windows operating system and that you’ve got an end video card and you’ve got at least 4GB of V Ram, you’ll also need about 10GB of storage space on your hard drive as well. I’m going to be showing you live and I’ve done a few of these tutorials for you guys and I found this one is the simplest one to follow.

There are eight steps in total and you need to make an account with the website HuggingFace co go and sign in there and make an account and then confirm your email address and with GitHub as well. So the links are below. So hopefully you are making those accounts now as I speak and you can pause this video if you go down in the description below. I’ve named all eight steps and they have the link to go to for each step. Although you’ll find this tutorial in a written format on my website which is promptwhuse.com. So the first step is downloading Python. Click the link in the description on number one and when you get to the page, scroll down to the very bottom and you need to click on Windows installer 64 bit. This will download that file to your downloads folder and then once that’s done downloading and you need to double click the file in your downloads folder to install the wizard. Ensure that you’ve got add to path checked that’s vital, otherwise this is not going to work. Okay, onto step two. So step two is downloading Git for Windows if it is called Git Bash.

So you go to the link in the step two description below and that’s gitforwindows.org and click the lovely download button here and again that will download to your demos folder. So once that’s downloaded, you need to run the installer and double click it and it will automatically run the installer for you. Just click next. That’s absolutely fine. And once you have finished installing that, on the last page, just click Launch Git Bash. So that will launch the programme for you. Step three. This is going to the HuggingFace co link in step three. And hopefully before you pause the video and made your hugging face account and you are logged in. And when you go to the link provided, you will see download weights here, near to the top of the page. So this is the latest release of Stable Diffusion. We are on one four, but you might be in the future. So it might say one five. So just click that and download that. So now we need to navigate over to your local drive line. In this example is MYC drive. You need to right click and create new folder. Name that folder, anything you want.

Called it FD for stabledusion. Just name it absolutely anything you want. AI, artwork, whatever.

This is where we are going to.

Be copying the repo into where we’ll be running Stable Diffusion. So make it something, you know, fancy and that you will remember what it is. We are now on to step number five. So that is going to GitHub.com. So that link will take you to this page and you’ll see near to the top this big green button that says code on it. Click this button and you will see a URL. You need to copy that URL and if you’re not running Git Bash like I said previously, to launch it, just search Git Bash on your PC and you’ll find it because that’s what we install. All you need to now type is all lower case Cdspace and then the drive that you just made that folder in, which mine is C and then the name of that folder, whatever you called it, I called Mine SD and then hit Enter on the keyboard. And now you need to type in Git, git space clone, git clone and right click and paste and that will pay for the link we just got from the GitHub website. Now you need to hit Return on the keyboard once again.

That’s cloned the repository into that folder that you have just made. That’s where we’re going to now run Stable Diffusion from we return back to the drive where you saved your folder to again, mine was C drive and what’s called SD and you will see a folder in there now. So we need to open that up and navigate down to models and then Stable Hyphen Diffusion should be in there. Now we downloaded that SDV one Hyphen four CKPT file and that should be chilling out in your downloads folder now. So go back to your download folder.

You need to right click and then cut and paste into that Stable Diffusion folder.

We are pretty much done. All we need to do now is come up to levels in that folder and you will see the batch file. Weboo user double left click to run that batch file. This will now install stereo diffusion on your PC.

Installing all the assets will take a few hours. It’s better to leave it running overnight. If you think it’s crashed or paused, just hit spacebar on the keyboard and that should get it running again. Do not click anywhere in the box because this will pause and freeze your installation. Once it’s finished, it will let you know it’s done, and you just copy the IP address that you’re given and paste it into your Internet browser. And there you go. That stable diffusion on your machine locally.

<p>The post How to Install Stable Diffusion on PC first appeared on Prompt Muse.</p>

<p>The post How to fix AI art faces : Midjourney, Stable Diffusion, Dalle 2 first appeared on Prompt Muse.</p>

You can try out different subjects, you can try out different styles, cyberpunk, steampunk, art nouveau, anything you can think of and add different artists to the mix and different mediums and styles and lighting. It blows my mind every time I use it and it’s only going to get better. This is an exciting time to come on board and learn how to write prompts. If you’re thinking, oh, this sounds very complicated, I don’t know where to start, start here. Literally, the secret is writing prompts is the same as writing. In Google Search, you are writing a text description of what you want to see and then hitting return and then midjourney. It brings you back an image result that it thinks you want to see based on your text description, exactly like a Google Search. So I’m going to break the prompt down into four simple, understandable components. So first you’ve got your concept and your subject. This is where the most weight is going to be at the beginning of the font. This is where you will define what it is. The next section is style. So you can style your artwork with a genre or a particular artist or mix in different artists.

You can use multiple artists and or different medium of art. So you could try photography, charcoal, sculpture. Just have a play around with all these different styles and you’ll be able to come out with some pretty cool images. So next is the quality inputs. HD stands for high Definition. You can add cinematic lighting. You may have seen Octane Render, which is a 3D renderer. The last and final part of your prompt is the composition. So how you want your output to look like. Do you want an Ultrawide shot? Do you want a wallpaper, like a desktop wallpaper. Do you want a macro shot or a specific headshot of your concept or subject? Put that in here. You can also put your aspect ratio so you can add the ratio size or the pixel size that you want your image. This will change your composition and sometimes gives you better images. So play around with the aspect ratio. Now, if there’s any words that you don’t understand on this screen, google it. Get used to all these different terminologies AI.

Art is not an exact science.

You can become a director and by using the correct prompts, you can get closer to the feeling or the vibe that you want from the image. So all of that in practical terms, it’s all well and good. So let’s put the theory into Practise. Let’s type in sitting black hat, wearing glasses, art by Art Germ, cute cinematic style, HD detail, octane render, and the aspect ratio of two by three. This is the result of our prompt. I did a couple of variations on this image and when I was happy with the variation, I upscaled the image and this was the result. And I think you can agree it’s pulling all the information from the prompt. It’s a black sitting cat with glasses on. It’s cute, it’s got a cinematic style, it referenced the artist nicely, and the aspect ratio is two by three. In this circumstance, Mid Journey has understood the brief and I know it’s not always a way. And you might have to reroll your gift closer to what you want. So you might be wondering what would happen if I just took all the fancy pants descriptions out and just put black cat in glasses.

Well, this is what my journey would give me. It is a black cat in glasses, but there’s no stylization and it quite literally is a black cat in glasses. So you can see how the prompt really influences the output that midjourney will retrieve for you. So hopefully, from this point forward, by watching this video, your prompt structure has improved. Now, to improve on your language, you can go and get inspired by heading over to the Midjourney.com website. And if you go below home and see Community Feed over on the left and click on that, you can see what images are trending, what images are popular, what images are new. What’s really cool is you’re on the standard subscription. If you click on any image and scroll down, ignore this text here, that’s not the prompt. If you click on the two dotted lines, three dotted lines here, and click on Copy Command and open Notepad, Word, whatever you got and just paste, you can see the exact command that they used for that image. Now, if you put this into Midgenit, you will not get a same image because it’s re rolling the dice. It’s not the same seed number, which I can explain in another video, but you will not get the same image, but you’ll get something similar.

But you can see that they have used pretty much all the structures that I explained earlier on in the video. So they put their main concept subject. Their artists actually use the same artist reference and what kind of design, what kind of quality they want from the image. As well, you may notice no watermarks. So if you want to subtract something from your prompt, you put in no glasses and that would remove the glasses from my cat. Or if you were doing a landscape, I’ve been hyping no trees and that would remove all the trees from your image. So that is a pretty cool trick. They also don’t want any borders, they don’t actually want any signature on their image. As you notice, some images from midjourney will show them swinging line a signature and you usually have to photoshop those out. But if you request no signatures, your image won’t have that in. That’s a really cool thing to learn as well. So, as I said, when you come over to the midjourney community, you’ve learned so much by looking at other people’s artworks. So we spoke about all the good things to put into your prompt and how to structure them.

There are things that you should not include in your prompt and those are banned words. You can go to the Journey Discord server and on there you can find the Rules channel. And on the channel the statement goes as follows do not create images or.

Use text fonts that are inherently disrespectful.

Aggressive or otherwise abusive. Violence or harassment of any kind will not be tolerated. No adult content or gore. Please avoid making visually shocking or disturbing content. We will block some text inputs automatically. So there you go. That’s everything you need to know in a nutshell. But if you are still unsure, just head over to the Rules section on the Discord Server and you will find their terms and conditions. I try and keep these videos as bitesized as possible because I know it is a lot to take in and I do have other videos that expand further on prompts, but I hope of every video I do, I improve and get you the information that you need clearly and concisely. I would really, really appreciate it if you follow my channel and subscribe.

And just before we go, I don’t want to say I have a prompt music Instagram page where we can interact over there and you can see prompts and images that I post. I pick a muse a day, so it’s worth following. And if there are any tips you feel I’ve missed out, please feel free to put in the comments section so other people can see and learn. Thank you so much guys, for joining me for another video and I’m looking forward to making another one. And that’s it for now.

Thanks a lot. Bye.

<p>The post How To Write Prompts  first appeared on Prompt Muse.</p>

Creating your character

Step one is creating the face of our character. So the first thing I’m going to do is head over to Midjourney. Here we will just create a front perspective of our character, showing their features clearly.

Like any good design, it’s good to have a plan and idea of what you would the overall style of the character to be, to suit your narrative. I want to start with the front perspective of my character, this is important as we will animating her so will need a good clear image of her features.

My prompt is:

/imagine Head and shoulders shot of Instagram model, with orange long hair, hyper-detailed.

Midjourny will give me an option of 4 images to select from. I really like variation 3 (V3). I select U3 which will upscale my chosen image. Now I have the front perspective of my character, I click on the image and right click and save onto my PC. We are now complete with Midjounry.

Consistent facial perspectives

I will now concentrate on getting multiple frames of my character which will enable me to build a model and in turn prompt her in any situation, environment and in any pose. A AI model is a collection of images on a subject that is then trained by AI.

I create a MP4, just simply by recording myself on my phone, making an array of emotions, ensure I keep my face steady. The dimension of this video is 410×412 px I also take my midjourney image and save that as 410×412 px

It’s important that I create multiple images of my character with an array of emotions to feed into the training dataset. To do this I will take the MP4 I created and esentiall skin it with the source.png within the Thin Spline Plate Model.

I name the video driving. Mp4 and the image source.png and upload to my googledrive (Top level/not in a folder), so I can easily upload it into google collab Thin Spline plate model

Thin Spline Plate Model

I open the Thin spline plate model within google collab worksheet:

https://colab.research.google.com/drive/11pf0SkMIhz-d5Lo-m7XakXrgVHhycWg6?usp=sharing

I run each cell in the notebook, ensuring I have a green tick before I move on to the next cell.

It is vital that you mount your google drive (This is just a fancy way of saying connecting) As this is where we are pulling out mp4 and source images from. To do this, simply select the folder icon and then click on the Google drive folder to add the mounted drive to your notebook, Run this cell and log into your google drive.

Step1: Setup

Step 2: Settings : Edit your source path: and driving video path to match the correct path to connect to your google drive.

You do this by locating your Gdrive from the left-hand folder hierarchy Content>drive>Mydrive. Find your Driving.mp4 video and next click on the 3 dots and select copy path. Simply now paste that path into the correct paths within Step 2. Once completed Run this Cell (Click play!)

Okay, so here’s the magic. You can now see the source image of the driver video and your final result. And you can see the face is going a bit weird when you turn too fast. So don’t do that in your video! You can turn to a certain degree, but it starts screwing up when it goes to the side.

Step 3: Run Thin-Plate-Spline-Motion-Model

Once the previous step has given you a green tick, proceed to run step 3 cell (No addtional work required here!) This will autmatically create another folder in our folder structure over ont he right hand side. Once this video has finished being upscaled, we’re going to run it through something called Gspgan, and you might have heard of that before. It’s a facial restorer.

So we’re going to then split it into frames and then run it through there. And it should make the face a lot nicer, but it sounds like hard work. It isn’t. We’re just going to press Play on this and it’s going to do it for us. So we’re onto step five.

Step five is now going to divide your video up into frames, which will allow us to run it through the G FPG facial restorer on each frame. So I’m just going to hit five and you know what to do by now. So we’re just going to wait for that to split the files. You can see here in real time. It’s creating those files into the frames folder and you can see all those files being saved there at any time.

If you want to save an image or a video, you just go to these three little dots here and click on Download. And that will download it onto your computer. So that has now finished while I’ve been talking. And if we go down, we’re going to be now running it through the facial restorer, which will basically increase the resolution on the image and make the facial features more appealing. So I’m just going to hit play on that.

And then we’re nearly onto step seven, which is our last step, I promise. But it wasn’t that easy. It wasn’t too hard, I hope. And if you did have any red errors, any problems, just put down in the comments. We have now completed all seven steps and your final video is over here and it’s called Out MP4.

You’ve got all your frames as well, which you can download, and your fixed frames as well. I’m just going to click the three dots and then click Download, and then that will download this to my local computer. So I’m going to show you the results of this video. Mine’s not going to be very good because I know I don’t my head away too much in the video, but you can see our output. I’m now going to take these frames and train them in stable diffusion and create a model that I can use to make prompts and prompt this character into any place.

So these are the final frames that I’ve saved from my colab notebook, and I saved them locally on my machine. What I have done is just delete all the frames that are all the same and just kept some distinct frames of facial expressions that I could use to train stable diffusion with. So this is how we’re going to get a consistent character. You might be thinking, I don’t have any body shots, but we’re going to fix that in the next tutorial. We’re going to be using out painting to add a body onto this character.

Now, if you’re not too interested in doing the body, you can go ahead and skip the outpainting tutorial and go straight into training stable diffusion with me. So now what we are going to do is a bit of out painting. So we’ve got plenty of headshots of our character and what we want now is more of her physique or her body shots. And to do this, I’m going to be using out painting, which reimagines areas that are around your character using a prompt. So for this, there are many ways of doing this, but again, my computer is not powerful enough at the moment to run it locally on my machine.

So I’m going to be using Google Collabs. If you go in the link below, you will see the Stable Diffusion Infinity collab link. So if you click on that, this screen is what you will see. And I’ve already run all my cells here, so I’ve pressed play on each one. But if you just start the setup and click Play, and that will install all the files remotely to the left hand side of this file area over here.

And then once that’s complete, then go to the setup of Stable Diffusion infinity so step three will continuously run. So it’s going to continuously run in the background. And what you’re looking for is when it’s loaded, you will see running on public URL. So we’re going to take that one, not the local URL because that means we’re using our machine’s GPU and I want to use the remote GPU. So I’m going to just copy the running on public URL and copy and paste that into your browser.

So when you’ve copied and pasted that link into your browser, you will get this screen here and all you need to do is get your hugging face token. Hugging Face Token is a unique identifying key to yourself. So we just go to the hugging face website and if you don’t know how to get to this page, just simply go to Settings. And then here on the left, click on Access Tokens. Super simple.

And click on New Token. And this will create you a new token or a key or whatever you want to call it and just give it some name that has some reference to whatever you’re working in. I’m just going to call my Infinity because I can’t sell infinity. And then I’m going to click on Copy and then go back to Stabledfusion Infinity and paste my token in that section. Now, I’m just going to select the stable diffusion in painting.

You can have the in painting an image image, but I’m just going to use In Painting for now and that’s it. And just click to set up. And this will now load your interface up. So now you will see a screen like this if you’re successful. And this is really, really simple to use.

So if you go to upload image, I’m just going to upload just any image of my woman’s face. Just scale it down slightly and I’m going to make sure enable Safety checker is off because that’s not safe for work checker. Basically, if you’re showing any nudeness, it will vendor a black box and we don’t want that. So for some reason when you’re doing a woman’s body, even if she’s not nude, it will flag it up. So uncheck that and you can do not say for work images or what it regards not safe as work images.

Once you’ve placed her in a position on the canvas you’re happy with just click Confirm and that’s what sets her into place. And you’ve got a prompt to a woman in green top. I don’t know if that’s a very good prompt, probably not. You can actually interrogate the image and it will give you what it thinks the image is. You can then adjust that prompt to be a bit more stronger.

But for now, I’m just going to set it as that. My sample size is six, my strength is zero. I’m going to put zero seven. My mode is CV Two underscore NS. Everything looks good and I’m going to just click the output button.

And this starts rendering the output. So you can see the processing bar down here and you can also switch back to Google collabs. And if you go to the bottom, because I’ve been working on this, you can see the percentage bar there as well. So that’s why that’s continuously executing in the background because it’s basically running this interface that you can see here. So as you can see, it’s giving me the woman in a green top, some sort of weird white thing there, but you can just paint that out.

I’m going to click Accept on that. You can cancel or run a retry. Remember, this is absolutely free. So you can just retry as many times as you like that you can move this generation frame around the screen to create more out paintings if you wanted to put her shoulder in or hair on top of the head. And once you’ve finished, you can then just go to Export Image here and then export your image as whatever you want.

And that saves it locally to your downloads folder. That’s a really cool way of getting the body. So once you’re happy with the set of images you’ve got for training this is mine. It’s not quite a good collection, but I’m trying to film this tutorial at the same time as doing this. So hopefully yours will fare better than mine.

We are going to now head over to a new notebook. This is a Dream booth, stable diffusion, and the link is down below. So we’re going to be using this notebook to train a model on our images that we’ve created. So everything has led up to this point. So what we want to do, you should be used to this environment.

Now we’re just going to cheque the type of GPU and VRAM and yep, I’ve got Tesla TV Four I’m running remotely at the moment. And then I’m just going to install this. So we just need to log back into our hugging Faith account and go to Settings and then access tokens and then want to create a new token. I’ll call this Dream and then generate the token copy that you’ve done this before, so it should be easy. And then paste that token into your hugging face token area and hit run on that cell.

Okay. And then we’re going to just install these XFORM moves here. I would say this to my Google Drive, but I don’t actually have enough space at the moment. This is the model you are running. We’re actually on two now, overnight, or yesterday we went into version two, but I’m going to keep it on version one here.

You can change that path if you want to use another version. And the output directory, so that will be up here. That will create the directory here. I’m just going to keep it to the WX, but you can call it Redhead Lady or whatever the name project is. I’m just going to leave mine as default for this demonstration.

Okay, so there’s some sections here we want to change. So our lady is not a dog, I’m going to name her a person and photo of a person.

OK, so we’re just going to run the sell here and that will create our folders that we’re going to just drag our images into. So if we go to the file directory, we should have a new folder oops into data, sorry, and ZWX. And that’s where I’m going to drag my training data into, which is all the images we’ve created. So I’m just dragging these locally off my machine and just throwing them in and uploading them and that’s just a message saying once the runtime is finished, it will delete the images, which is fine if you read this one you can upload from your computer. But as we frame them into the file, we don’t need to do that.

So we can then just go to the next so just run this one, but I need to change that to person, not dog. And I’m going to keep all these settings as the same and then just run that cell. That last cell took a rather long time, so I hope you made yourself comfortable while that was running. So we’re now moving on to the weight cell. I’m not going to change that, so I’m going to keep that as it is.

I’m not going to run the grid. So this next section is what we’re going to do at the end. This is converting the weights to a CKPT package or model to use in web UIs like automatic one one. So that’s basically going to be our output that we’re going to load into our local stable diffusion to write our prompts. You can use this notebook to do that in, but you can do it locally on your machine, which is a lot easier and a lot better than doing in here.

But that is going to be our output. We’re not going to run that cell just yet. We can do that at the end. The next section we have is the inference. So I’m going to run that.

And these cells now from now on are a lot quicker than any of the ones above. Okay? So after inference, we are going to the sea. So you can set random seed here for reproducibility. I’m just going to press play.

Okay, so photo of ZW x in a dog in a bucket, that’s not what we’re creating. We’re just going to do ZWX face is that what my prompt was, which is a photo of a person. So ours is the WX person. So negative prompt is for example, if you want the person to have long hair, you’d put negative prompt in short hair, so it avoids short hair in any of the images. I’m not going to do anything for that at the moment.

Number of stance pulls. I’m going to keep it four guidance scale. You can lower that, but I’m going to keep my 10.5 and fahrenheit. Yeah, so I’m just happy with all those settings. So I’m going to run and see what this gives me.

Hopefully this works after all that. So this is pretty speedy. So it should give me somebody who looks like my Redhead model that I created in mid journey. There you go. There she is.

So she really looks like my training images, which is great. So if I go to my data and go to my ZWX and open Redhead, we can compare our training. Oh, that’s not a good thing. Let’s get a better one. There we go.

We can see our training data compared to what we’re getting, which is really good. It’s a really strong likeness, and now we can go up and save that model. So if we now go back up to convert weights to CKPT to use in the web UIs, we can now save this as a model to load into our local stable diffusion installation. If you have got that, if you haven’t got that on your machine and have more than 4GB of VRAM, you can run it. If you have less than that, I wouldn’t even try.

I’m doing a new video on installation of Stable Diffusion 2.0, because that has just come out overnight, so I’m very excited to do that. So the video will appear here once it’s done. If it’s not there, please badger me in the comments for that because I need to update mine and see the difference. So I’ve got to test that. So that now has saved my file, and it tells you, here the path it saved it to.

So let’s find that. Here we go. Up one level and into contents and model stable diffusion weights, the WX 800. And there she is, model CKPT. I am literally losing the ability to talk, so I’m just going to download that and I’ll show you how to save that into your local stable diffusion.

Sorry, I’ve gone completely nuts. So, yeah, we have now got our trained character. So here’s a few more prompts that I put in. So you can see sometimes it gives you a bit of a crazy image. And there you go.

She’s holding a glass of wine. So compared to the training image, you can see it’s got really good likeness. And now I’ve done photo of ZW, ex person, cyberpunk style. And as you can see against the training images there. So the CKPT file is going to take a while to download.

It’s a couple of gigabytes, but once it’s downloaded and you have stable diffusion locally installed on your PC, just go to that file where your stable diffusion lives, and then go to Models and Stable Diffusion, and then just put the CKPT file in here along with all the others. And then if you just go back up like this and click on Web user to launch your Stable Diffusion you have downloaded your CKPT file and put it into the Stable Diffusion Models folder. You should be able to see it in the dropdown list here that this is Mine. Here something very important. Now, when you are prompting your character, you need to write the name that you wrote in the instance when you were training your character within Dream Booth.

So for Mine, I trained Mine on being a person right at the beginning and then scroll back at the tutorial. And the name was ZWH. I think it was just left as the default. So her name is ZWS and she is a person. So I can change the beginning and end of this sentence, but I just have to always ensure that I indicate it’s ZWX person to get the strongest likeness to my trained character.

So now you can make any image you like of your character. You can make her into a comic strip, you can make her into a painting, you can make her into an Instagram model. Whatever you want to do, make sure it’s legal and completely above boards. And this is it. You’ve done it.

And as always, that will do it. Hit that notification button and subscribe everyone like. Thank you. Bye.

<p>The post Consistent AI Characters in any pose: Tutorial first appeared on Prompt Muse.</p>

Stable Diffusion is a state-of-the-art machine learning model that can generate realistic and detailed images from natural language descriptions. In this guide, you will learn what Stable Diffusion is, how it works, and how you can use it for your own creative projects.

What is Stable Diffusion?

Stable Diffusion is a latent diffusion model, a type of deep generative neural network that can learn the distribution of complex data such as images. Stable Diffusion was developed by the CompVis group at LMU Munich, and released by a collaboration of Stability AI, CompVis LMU, and Runway with support from EleutherAI and LAION in 2022.

Stable Diffusion can generate images conditioned on text prompts, such as “a blue cat with green eyes” or “a surreal landscape with a castle and a dragon”. The model can also generate image-to-image translations guided by a text prompt, such as “make this photo look like a painting” or “add a rainbow to this scene”. Stable Diffusion can also perform other tasks such as inpainting, outpainting, and style transfer.

How does Stable Diffusion work?

Stable Diffusion works by gradually transforming a random noise image into a target image, following a diffusion process. The diffusion process is a stochastic process that simulates the movement of particles from regions of high concentration to regions of low concentration. In Stable Diffusion, the diffusion process is reversed, so that the model starts from a low-information image and adds more information at each step, until it reaches the desired image.

The model learns to reverse the diffusion process by using a neural network called the denoiser. The denoiser takes as input the current noisy image and the text prompt, and outputs a prediction of the next image in the diffusion process. The denoiser is trained by minimizing the difference between the predicted image and the ground truth image from a large dataset of images and captions.

can you use Stable Diffusion?

Stable Diffusion is an open-source model that you can access and use for free. There are several ways to use Stable Diffusion, depending on your level of expertise and your needs.

If you want to try Stable Diffusion online, you can use the official website https://stablediffusion.fr/, where you can enter your own text prompts and see the generated images. You can also browse the gallery of images created by other users and artists, and get inspired by their prompts and results.

If you want to use Stable Diffusion on your own computer, you can download the code and the model from the GitHub repository https://github.com/stabilityai/stablediffusion. You will need to install some dependencies and follow the instructions to run the model locally. You can also modify the code and the model to suit your own needs and preferences.

If you want to use Stable Diffusion in your own applications, you can use the Runway platform https://runwayml.com/, where you can integrate Stable Diffusion with other models and tools, and create your own workflows and interfaces. You can also use the Runway API to access Stable Diffusion programmatically from your own code.

<p>The post Whats is Stable Diffusion? A complete guide first appeared on Prompt Muse.</p>