How to Create a 3D Scene Using Blender and Fspy

Transcript

Ai community, I have confession to make. The creation of endless AI images is boring me until now. Today, I’m going to be showing you how to take your AI images that have been created in Stable Diffusion, Darley, Midjour, whatever program you’re using, and turn them into an ultimate movie set, which means you can configure how it looks, get a camera, go in, add characters to the scene, create an animation from your 2D images. We don’t want AI to be the creative director of our stories that we want to tell. We want to use AI to enhance our workflows. And that’s what this channel is all about, creating workflows that actually work. Today, we’re going to be looking at how to create an environment from AI generated images. And then in the next video, we’re going to look at how to add characters into our environment. So please hit that notification bell and subscribe because this is going to be a series. So we’re going to start off with the environment workflow. So the first thing we want to do with our AI generated image is upscale it because what comes out of midjour is quite low resolution.

Same with automatic 111. You might not actually have the RAM or GPU to be able to upscale that on your PC. So I use something called Neuro AI. This is absolutely free and to be honest, it’s a godsend. There’s no download. It is what it is. You upload your image and within seconds it gives you a high resolution image. So when we’ve got our upscale image, I’m going to take it into F Spy, which again is amazingly free, which will create us a camera that matches the image perspective. And then we simply import our F Spy camera into Blender, which will be then projecting our map from the camera onto very simple, primitive objects. It’s really an effective result that we get. Step one, upscaling our image. I want to increase the resolution of my image that Midjour created because in the background, if I’m going through the scene, you’re going to see as you go through the scene, the image lose resolution and just see big chunks of pixels, and we do not want that. So we’re going to use an upscaler. And today I’m going to be using AI Neuro. Currently, you get five free credits a day to upscale your image.

So I’m going to click on Download and download that image. So we’re now going to take this image into F Spy to create our perspective or projection camera. Step two is installing F Spy, which will create us a camera that we can then import into Blender. Each image that you bring into F Spy will be completely differentand have different perspective lines. But what it allows you to do is ultimately create a camera that you can then model in Blender from. There are two zip files on this website that we want to download, the first being the program and the second being the installation plug in for Blender. If you head over to the F Spy website, there’s a big green Download button, and that’s to install the actual program onto your PC. You’ll be taken to a GitHub page where you just need to download the extension with win. Zip at the end if you’re running on Windows. And if you download that and unzip that onto your PC, you’ll be able to see F spy. Exe file, which you need to double click in order to run the program. Once that’s installed, you need to head back to the F spy website.

And if you scroll down from the main page, you’ll see the official F Spy importer add on. This is the zip file which we’re then going to install directly into Blender. Download that file by going to this big green Kodabutton over here and come down to where it says Download zip and download that zip file. If you just fire up Blender and go to Edit, Preferences and go to Install and just find that F Spy Blender Master, click on Install Add On, no need to unzip it or anything, and you should find it in your search bar up here. Just make sure it’s checked in the checkbox there. Go to the hamburger icon and save preferences, and you’re good to go. When you go to File and then then to Import, you should see. F spy here. So now, Minimize Blender, and where you unzipped the first F Spy folder, just navigate to the f spy. Exe and give it a double click and that will launch F Spy. So you can simply drag and drop the image you got out of Midjour here or you can go up the file and open image. This is F Spy and the first time you use it, it does look a bit intimidating.

But do not worry, all you need to focus on pretty much is this gizmo here. This is the most important thing in F Spy. We want each corresponding axis to line up with our image as perfectly as possible. The X axis is the horizontal line across the image. So you’ve got Z, X, and Y. These lines here are what we’re going to do e are going to place manually to help the program understand what is Z, X, and Y. You can see our Y axis, so we need to mark the vanishing point. If we put this green line here, which notes the Y axis, and then this green line here to the other side, you can see it’s creating a vanishing point at the end of this road. Now, it’s quite hard to see where you’re laying these lines down, so you need to come over to the left hand side and uncheck this dim image box here. And then that will help you position your lines further. You can also hold shift on the keyboard and position the lines, and you’ll get this lovely big magnifying glass that will help you a little bit more.

So as you see, while I’m lining these lines up, this gizmo in the middle, which I said is vital, is lining up as well and positioning itself correctly. You can see my X axis off. I want that to be horizontal with the floor plane. So I’m going to put my Z axis here. I’m just going to find basically a vertical line on the drawing. So it’s important to line up, for instance, my Z axis parallel to one another so the program can measure the distance between them. That is looking good. And if you check my gizmo, the Z axis is pointing straight upwards in line with everything else. So it’s looking good so far. And to check that your lines are in the right place, if we go down here to 3D Guide to the drop down menu and go to X, Y, Z grid. You can then place your grid in and just make sure everything is lining up. You can switch to box as well and just check that your box looks like it belongs in that perspective. You can also line it up to any of the lines in the image and just double check that everything is lining up nicely.

If there’s anything out, you can adjust these lines further to get it correct. This is the final position where your projection will load up in Blender. So it’s important to try and centre this gizmo as well as possible. So that’s all looking good and I’m ready to import this into Blender. So I’m going to go to File and go to Save As and Save This. And we’re going to now bring this camera projection into Blender. Step three, adding projection camera and material to 3D geometry. So I’ve just opened up a Blender project, and I’m now just going to marky select and delete any objects that are in the scene. And then I’m going to go up to File and import. You should have F Spy here. If you don’t, go back to the beginning of the tutorial and install it. So I’m going to click on that. I’m going to navigate to the F Spy file, which was cyber. Nvi. F spy for me. I’m just going to click on Import F Spy Project File. You can see here straight away, it creates an F Spy camera up here in the scene collection, and it’s automatically projecting that scene.

Now, on your keyboard, if you press zero on the key number pad, you can see that the camera is not projecting onto any plane at all. It’s actually projecting onto itself. It’s quite clever. So I’m going to press zero again on the number key pad. This is a super easy modeling. We’re just going to be using a plane. So to access your planes, hold down shift and A and go to mesh and across from mesh, select plane. So I’m going to create another window so you can see exactly what I’m doing. From the top left of this window, when I get the cross head, I’m just going to left click and drag and that creates you another window. And in this window you can see my plane and my projection camera there. Now I’m just going to select this plane and go to Object Mode up here and Edit Mode. I’m going to click on Edge Mode, which is here. I’m going to then select the back edge, which is here and press G and Y. And then I’m just going to extrude that back edge right to my vanishing point down there. So this is what it looks like so far.

Remember, the Y axis is from the viewport right down to the vanishing point. I’m now going to come back down to Edit Mode and I could press S for scale and then X for scaling on the X axis. So it will just scale along the horizontal line. So I’m going to select both edges on either side of the road and then press E to extrude and then Z to make sure that it’s on the Z axis. I’m just going to come up there and extrude up to the pavement. I’m now going to select the left side and again, repeat that process. Press E to extrude and then X so it just snaps to the X axis. And again, once more, E to extrude and then X to extrude on the X axis. So I’m going to click on both edges of the sidewalk here and then press E to extrude and then Z so it snaps to the Z axis there. And I’m going to come right up there to the top of the buildings. And I’m just going to go to Edge and I’m going to then go to Bridge Edge Loops. And then again, at the back, I’m going to do the same, select both edges and then click on Bridge Edge Loops.

That is now pretty much all the modeling we need to do. If we come out of edit mode and come back to Object, we need to go over to the Shading tab. So we want to apply the material. So once in the Shading tab, ensure your object is selected and go to New. We just want to delete the principal BSDF node by selecting it and hitting delete. We want to select the material output and on the keyboard, hold down CTRL and T. This activates your Node Wrangler. If nothing happened when you press CTRL and T, your Node Wrangler is not enabled. So I suggest you go and enable that. And to do that, you go up to Edit and down to Preferences. And just type in the search bar here NodeW rangular. And all you need to do is just make sure that box is checked and go to this hamburger icon and click Save Preferences. And then just repeat that process again. Just click on Material output and hold down CTRL N T on the keyboard, and these should come up here. Now in the image texture, this is where we’re going to load our upscaled image, it will look a mess when you import that in, but do not worry, we’re going to fix that now.

So if you come over to the Spanner icon over here and from the Add Modifiers drop down list, you want to go to Subdivision Surface. So give that a click, and it will be selected default as cat mall clerk. But we want to switch that over to simple. And then on the level viewport, we want to add five onto there. And then on the render, we want to make that five as well. So next we want to go back up to the Add Modifier drop down and come over to UV Project, which is there. Now under the UV Maps section here, just select that box and select the only UV map that should be in the scene. And then come down to Object, and then under Object, select your. Fspy camera. What we need to do is put the height and the width of our original upscale image into here. Just go back to your image, right click, go to properties and details, and your resolution or your dimensions will be there. So mine is 4196 by 2796. So yours will probably be different. So I’m just going to go in and type in 4196.96. Now there’s a really annoying Edge Repeat round there, and we can change that because that is currently set to Repeat in the settings.

So if you come back down to your image node and come down where it says Repeat, and hit that drop down box and select clip. That will give you a black area, so that makes it a lot easier to see your actual image. As you can see, we’re slowly building up our 3D scene. Now, if you click on the projection camera and move it, bad things happen. You do not want to do that. So what we need to do is just make a copy of that camera. So hold down shift and D and then right click on your mouse and that will create a copy. Now, if you go back to the original camera on the Object properties here, we need to just lock that camera into place. Just hit these padlocks and it will not move. Now we’re going to give our new camera a name and I’m going to call it Movie Cam 1. With Movie Cam 1 now selected, we just move that. Then right click on the camera and click Set Active Camera. So this is now our camera that we’re going to be using to go through our scene. So when you go in and out of your scene, just make note of what is distorting.

So you can see these bins on the right are distorting and this plant and the neon signs. I’m going to bring this tree in here as a plane and then this neon sign, I’m going to use UV projection so you can see both methods to see which suit your scene best. In this step, I’m just removing the background from the plant tree shrub thing. You can use any free software to do this. I’ve put some links in my description if you do not have Photoshop. So the first thing I’m going to do is right click on my background and click layer from background. Okie dokey. I’m going to use this great tool from Photoshop, which is the Object Selection tool. And then just simply select the object you want to select. And voila, it creates a selection of that specific object. So I’m going to press CTRL and J on the keyboard, and that just transfers the selection to another layer. So I’m just going to call that plant. And then I’m going to right click on my plant layer, duplicate layer, and go to documents, and then New, and then OK, and then to image, and trim that baby down.

I’m going to go to File and export that baby out of here. So I’m exporting it as a PNG, and I’m going to bring that in as a plane into Blender. I hope that’s plain and simple. So if we head back to our Blender scene, we can import our plant as a plane. So if you hold down shift and A on the keyboard and then go to image and then across to images as planes. We then want to navigate to our planned file that we just exported as a PNG outside of Photoshop. So the material settings, we need to ensure that it’s set on admit, and then click on import images as planes. And there she is. We have our plant. So I’m just going to press G and then Y and then push her back on the Y axis and just position her over here. Give her a little bit of a scale up there. And you can see there, the left side is clipping into the original wall. So we want to bring it out slightly and just set it roughly where the original plant was. It doesn’t have to be in the exact same spot.

And we’re just going to then click on the click on our movie camera and move her, GY, forwards. And as you can see, we got the original stretching of obviously the neon light and the plant going on. We are actually going to use in painting in a moment to remove those. So method number two, I’m going to project onto this neon light. And in order to do that, I’m going to make a square or rectangle object for that neon light and just grab a plane. And then I’m just going to simply position that plane where that neon light is. With our object selected, we’re going to go to Object Mode, Edit, and then on the keyboard, just press A. This will select all the faces. And then on the keyboard, just press U. And then from this menu, just select Project from View. And from the material properties, either assign the original background material or create a new material, base color, image texture, open, and then again select the original background. And as you can see now, if I come out of the camera mode, you can see we actually have a 3D object. You can do that with multiple objects in your scene, especially if your character is interacting with them, walking behind them.

It usually works best as a 3D physical object, but you can also use a plane technique for objects in the foreground or the background. We obviously now want to get rid of the duplicates in the background that are on our scene. So you can see our neon light and our really stretched elements in the background. And I’ve got a super, super, very cool AI tool for you to use for this. It’s called Runway, and I can see myself using this lots and lots in future tutorials. If we head over to Runway. So this is Runway. This is where we’re going to be essentially using their tools to do image to image in painting. My GPU currently is dying a very slow death and running things on my PC is not the way forward. Having access to all the AI tools in a browser is insane. We’ll be using this erase and replace tool. So simply upload the image that you want to use and use this big purple brush to paint out what you don’t want to see in the scene. I’m going to start off with these bins. So I’m going to just type in sidewalk with closed garage doors.

Fingers crossed this will work and that will magically generate a better image to work with. And here we go. It’s created a sidewalk with closed garages. That is pretty neat. Let’s have a look what it’s given me. So it’s given me a couple of options and I’m just using the arrows just to rotate through them. This probably best represents the scene, so I’m going to click accept. So now I’m going to just quickly go through this entire image and just remove elements and replace them with what I want to see using the prompt feature. Once we have finished removing all the bits we don’t want in our image, we simply just go and download that to a Download folderand head back into Blender and upload that into the background. So see you there. Now we’re back in Blender, just select your alleyway object and then go to material properties down here. We just want to replace the image with our new runway image that we just downloaded. As you can see, it’s all coming together nicely. I’ve just switched over to my movie camera. Remember, that’s the only one we’re going to move. I’ve added a keyframe at the start of the animation, and I’ve moved her right into the scene and back again just to check for any items or objects or materials that are stretching.

But it’s looking pretty good. So we got our plant there and our 3D object. You might remember in the original image, we had wires in our scene. I’m going to recreate those wires quickly because it’s quite nice to mix the 2D and 3D elements together. I’m going to hold down a shift A and go to mesh and add a cube. And this cube is literally just going to be where the starting point of our wire is going to be. And just going to scale that there. And then I’m going to shift a D to duplicate and then right mouse click to place. And then just put that there. And then hold down shift and select both of these Cubes. So with both Cubes selected, I’m going to hold down shift and A to open up our menu and come down to the second option, which is curve, and then come down to knots and then select catenary, catenary, catenary, catenary. I’m sure someone’s going to correct me in the comments there. And click on that, and you can see it’s created our wire straight away. We actually get an optional menu here, which we can actually adjust the drop of the wire.

We can also increase its resolution and its actual thickness as well. So we actually do want to see it in the scene, so we want it quite fit. You can go ahead and use that to add multiple wires to your scene. Let’s take a look at our 3D scene. As you can see, the geometry is super simple, and this could be put together in five minutes or less once you get the workflow down. So if I hit zero on the keyboard and change my material shader so I can see everything in the scene, if I hit space bar on the keyboard, you can see I’ve added two key frames to this camera and it’s just simply moving into the scene. I’ve also added a ShakerFly camera effect, which is super cool. And the plugin is in the description below and is absolutely free and super easy to install. You just go to edit, preferences and install the zip. The ShakerFly camera, once installed, will then appear in your camera object properties under Camera ShakerFlyer. There are so many cool settings in this. This guy who created this created all different scenarios, so walking or if you’re on a bike.

So this is a really cool effect to add to your camera. Also, I’ve enabled a depth of field, which is obviously included in the Blender itself. You don’t have to install this. And you can actually set the distance of your depth of field or a focus object. So if you have a character in your scene, you can make the background blurry behind them and have them in focus. Part two of this next series is adding our character into the scene. So please hit the notification and subscribe so you get that video. I hope you can take some techniques away from this video. I tried to keep it as simple as possible. So if you’re new to Blender, hopefully this is a nice introduction to using it. And of course, it’s not a perfect technique, but remember to get our stories and to get our animation out there. We don’t need it to be perfect. Perfection is the enemy of done, or something like that. If you want to add me to your social media, I would absolutely love that. My Instagram is @prompt muse, my Twitter is @prompt muse, and my Facebook is @prompt muse.

And of course, I have the prompt muse. Com website where we have started an absolutely free weekly newsletter. The newsletter, as well as all the online resources on the promptmuse. Com website is absolutely free. Just as a massive thank you for you subscribing to this channel and hopefully the newsletter as well. And thank you guys to every single one of you that comment in the comments section below of this video and all the other videos. I read every single one. Thanks so much for coming on this journey with me. And hopefully we’re going to have some great times and some innovations along the way. With that said, thank you so much and that will do it. Bye bye.