Mesh painting is the ability for players to paint on objects in-game. Examples of mesh painting are the goop in Super Mario Sunshine, the gels in Portal 2 and the ink in Splatoon. It can be used as a gameplay element or it can be purely for visuals. Either way, mesh painting opens up a lot of possibilities for gameplay designers and artists.
Although the previous examples are all pretty much the same effect, you can use mesh painting for many other effects too. For example, you can use it to spray paint objects, render wounds on characters or even let players draw their own character’s face!
In this tutorial, you will learn how to paint onto a skeletal mesh. To do this you will:
- Unwrap a mesh into its UV form
- Use the hit location from a line trace to sphere mask the mesh
- Render the unwrapped mesh and sphere mask to a render target using a scene capture
- Use the mask to blend between textures in the character material
Note that this is not a tutorial on vertex painting. Vertex painting is dependent on mesh resolution and can not be changed in-game. On the other hand, the method in this tutorial works independently from mesh resolution and also works in-game.
- Part 1: Painting With Render Targets
- Part 2: Deformable Snow
- Part 3: Interactive Grass
- Part 4: Mesh Painting (you are here!)
Start by downloading the materials for this tutorial (you can find a link at the top or bottom of this tutorial). Unzip it and navigate to MeshPainterStarter and open MeshPainter.uproject. If you press Play, you will see the character that you will be painting on.
Just like the snow and grass trails tutorials, this method also requires a scene capture. To save time, I have already created a scene capture Blueprint for you. If you’d like to learn more about the capture settings, check out our tutorial on creating snow trails.
First, let’s look at how you can paint on a mesh.
In most cases, the mesh you’re working with is already UV mapped. So the obvious thing to do would be to create a mask using a render target and then apply it to the mesh. However, generating a mask directly on the render target (using Draw Material to Render Target) will usually result in discontinuities across UV shells.
Here’s an example of a cube’s UV map and a sphere mask texture:
And here is the mask applied to the cube:
As you can see, a 2D sphere mask does not wrap around corners since it doesn’t take the geometry into account. To generate a correct mask, the sphere mask needs to sample the world positions of the mesh instead. But how do you access world positions when using the Draw Material to Render Target node?
If you’ve been looking at methods on mesh painting, you may have come across Ryan Bruck’s video on character damage using render targets (the method in this tutorial is based on his method). In the video, he successfully generates 3D sphere masks and accumulates them into a render target. He is able to do this because he creates a render target to store the mesh’s world positions which he can then sample using a sphere mask. Let’s take a closer look.
There are four steps to this method. The first step is to "unwrap" the mesh in question. This is simply moving all of the vertices so that you get the mesh in its UV form. For example, here are the character’s UVs:
Here is the character after unwrapping in Unreal:
You can unwrap a mesh by applying some simple World Position Offset math (which you’ll see later on).
After unwrapping, the second step is to encode the world positions of the mesh into a render target. You can do this by setting the unwrap material’s color to Absolute World Position and then using a scene capture to capture the unwrap. Here is what the render target would look like:
The third step is to create the sphere masks. Now that you can access the mesh’s world positions, you can sample them in a sphere mask. You can then draw the sphere mask directly to a second render target.
The final step is to use the mask in the character’s material to blend between colors, textures or materials. Here are steps three and four visualized:
Now, let’s look at the proposed method.
While Ryan’s method works, it requires:
- Two render target draws. The first is capturing the unwrapped mesh. The second is accumulating the sphere masks.
- One render target to store the world positions
- A render target to accumulate the sphere masks. You will need a separate render target for each actor you want to paint on.
The method in this tutorial discards the second draw and world position render target. It does this by combining the unwrap and sphere masks into one material (the unwrap material). It then captures the unwrap using an additive composite mode to accumulate the sphere masks.
One thing to note is that both methods work best when the mesh does not have overlapping UVs. Overlapping UVs will cause pixels sharing the same UV space to also share the same mask information. For example, you have both hands UV mapped to the same space. If one hand becomes masked, the other hand will become masked too.
Now that you know the method, let’s start with creating the unwrap material.
Creating the Unwrap Material
Navigate to the Materials folder and create a new material. Name it M_Unwrap and then open it.
Next, change the following settings:
- Shading Model: Unlit. This is to make sure the scene capture doesn’t capture any lighting information.
- Two Sided: Enabled. Sometimes unwrapped faces can be facing the other way depending on how the mesh is UV mapped. Two Sided will make sure you can still see any flipped faces.
- Usage\Used with Skeletal Mesh: Enabled. This will compile the shaders necessary for the material to work on skeletal meshes.
Up next is to unwrap the mesh. To do this, create the setup below. Note that I have already created the CaptureSize and UnwrapLocation parameters within the MPC_Global asset.
This will UV unwrap the mesh to the specified location at the specified size. Note that if your mesh’s unique UV map is on a separate channel, you will need to change the Coordinate Index of the TextureCoordinate node. For example, if the unique UVs is on channel 1, you would set Coordinate Index to 1.
The next step is to create the sphere mask. For this, you will need two parameters: the hit location and sphere radius. Create the highlighted nodes:
This will return white for pixels within the sphere mask and black for pixels outside. Don’t worry about setting a value for the parameters since you will do that later in Blueprints.
It is important that you set the Absolute World Position node to Absolute World Position (Excluding Material Shader Offsets). This is because the pixel’s world position will change due to the unwrapping. Excluding material shader offsets will give you the original world position before unwrapping.
That’s all you need for the unwrap material. Click Apply and then close the material. Up next is to apply the material to the character to unwrap it.