1 Comment

Vegetation improvements

Up until this point I’ve been using a modified version of the excellent L Trees library to generate procedural trees from XML settings files. This was flexible in terms of the variety of trees I could create, but they weren’t terribly realistic. This was especially true in the way that leaves attach to the branches. Furthermore the trunks tend to have a high poly count (though this can be controlled to a large extent in the settings files). A couple of trees I made looked like:

After looking into wind animations as per my previous post, I started looking for other plant sources and came across some nice tropical plants on turbosquid.com. They seemed to be nicely modifiable for the wind animations. Then I found some other tree models which I am less pleased with, but may still work out in the end. I’m trying to figure out the look I want. The L-Trees don’t look anything like the tree models I found (and the tropical plants look different yet).

L-Tree on the left, hand-crafted model on the right.

The hand-crafted model looks more shiny even though it has the same specular components. I think it just has to do with the angle of the leaves. The L-Tree leaves all point somewhat unrealistically up (this is a modification I made to the L-Tree construction, as it was even worse before since they always faced the camera). The hand-crafted model’s leaves just happen to angle more towards the sun.

I’m still trying to determine the right look. I want them to look good, not necessarily realistic. They don’t even need to be plants that actually exist in the real world.

Anyway, the tropical plants were the most interesting models, so I’ll go through some details of how I plan to render them.

Leaf Culling

One thing I had never addressed with the L-trees was proper culling of the leaves. We don’t want to cull backfaces since the leaves are just planes and we need them to be visible from both sides. In practice this didn’t matter since the trees are mostly viewed from above and the leaves’ normals pointed up. But with a larger variety of plants now with leaves facing every which way, I needed to address this issue.

Different lighting on each side of the same leaf.

If we turn culling off without changing anything else, we’ll get improper lighting. The lighting calculations are mainly done based off the normal. No matter which way the leaf faces, the world normal will be the same, even though we may be looking at the underside and the lighting should be different!

One option is to make 2 draw calls, one with clockwise culling and the other with counterclockwise culling, and each with a different shader parameter that indicates whether or not to flip the normal for lighting. Introducing a second draw call is undesirable for performance reasons though.

Another option – which I actually went to the trouble of implementing – is to duplicate the vertices but flip their normals. Now we’ve doubled the number of vertices we need to process though, so this isn’t ideal either (though it was my choice before I realized the obvious solution).

A third option, rather terrible, is to add face normals to each vertex (instead of just vertex normals). The face normal can be transformed into view space in the vertex shader, and we can use that to know which way the leaf is facing (you can’t do this reliably with vertex normals since the direction they face may change across the width of a triangle). This option is terrible of course because it means vertices cannot be shared among the triangles. This would roughly triple or quadruple the number of vertices that need to be processed for a plant. Not acceptable.

Of course, I didn’t realize the obvious solution until I had already wasted time on the others: The pixel shader has access to a VFACE semantic that indicates if the current triangle being rasterized is back-facing or not. So we just flip it based on the sign of the VFACE scalar.

float4 PixelShaderFunction(VertexShaderOutput input, float vface : VFACE) : COLOR0
    float3 normal = normalize(input.Normal) * sign(vface);

    // do lighting equations with the normal and return color, etc...

Alpha-testing artifacts.

Vegetation and foliage are commonly modeled as planes or other coarse geometry with an alpha map applied. The background shows through where the alpha map is transparent (e.g. surrounding the leaf image). This is usually done with alpha-testing.  Alpha-blending looks better, but is impractical (each leaf would need to be sorted and drawn from back to front). Alpha-testing on the other hand works on a per pixel level which avoids issues with the z-buffer. The results are a little blocky but can be smoothed with MSAA (multi-sampling) or for deferred rendering by post-processing algorithms that attempt to detect and smooth edges (such as MLAA or FXAA).

Alpha-testing is implemented with a clip function in the pixel shader. Pixels below a certain alpha threshold are abandoned.

	output.Color = tex2D(TextureSampler, input.TexCoords);
	clip(output.Color.a - AlphaTestValue);

This AlphaTestValue can be tuned depending on the look desired and the nature of the source texture. For instance, here is the same plant with three different AlphaTestValues:

Different alpha-testing thresholds

Lower values may result in dark lines around the edges (ideally you shouldn’t  be using texture with pre-multiplied alpha for this – that will make the dark outlines worse) since they leave in more pixels around the fringe of the leaf image. So why not just choose a high value? For instance, a value of 0.9 will discard all pixels with an alpha value of less than 0.9.

The problem comes when the leaves are viewed on edge. When that happens, the entire width of the original leaf texture is spread across a tiny number of pixels. This results in the GPU choosing a very small mip level. When sampling from a 1×1 or 2×2 or 4×4 mip level, alpha-testing kind of becomes meaningless. Entire sections of the leaf will be opaque where they should be transparent, and vice-versa.

Artifact from the GPU choosing a small mipmap.

The above image shows a leaf being viewed edge on. It ends up “extending” into space where it should not. We could avoid this by limiting the mip level that is chosen. This can be done in DX10 (or 11?) but not in DX9. Another alternative is just not to generate the smallest mip levels. This should work in native DX9, but for some reason XNA doesn’t let you do this. It requires either no mipmaps or the entire chain of mipmaps down to 1 x 1. So we can’t use this technique.

My initial attempt at fixing this was to make the smallest mipmaps completely transparent. That should eliminate this black line coming out of the leaves, I thought.

It did, but it introduced another artifact, which should have been obvious:

Now solid parts of the leaf , when viewed edge on, are transparent when they should be solid!

So really the only solution is to limit the mip level being chosen. And the only way to do this in XNA is to calculate the mip level manually and sample using tex2Dlod function that lets you explicitly specify the mip level.

float2 TextureSize; // Pixel size of the texture being sampled.
#define SMALLEST_MIP 4
float GetMipLevel(float2 iUV)
	float2 dx = ddx(iUV * TextureSize.x);
	float2 dy = ddy(iUV * TextureSize.y);
	float d = max(dot(dx, dx), dot(dy,dy));
	return min(SMALLEST_MIP, 0.5 * log2(d));

And in your pixel shader:

	float mipLevel = GetMipLevel(input.TexCoords);
	output.Color = tex2Dlod(TextureSampler, float4(input.TexCoords, 0, mipLevel));

Unfortunately this increases the complexity of the shader by about 10 instructions (in my case, going from about 50 instructions to 60). On my relatively crappy graphics card, it costs about 1ms per 1280 x 720 frame filled with lots of vegetation. It does address the problem though:

Mip level artifact fixed by manual mip calculation on the right.

Leaf color variation

Trees often have leaves of different colors. This is especially noticeable in fall on deciduous trees, but is commonly seen in other plants.

This can be done by applying a tint to the leaves. The original implementation I used along with L-Trees did a per pixel hue change. This can be done by “rotating” a color just like you rotate a vector with a matrix. In the shader this ends up being three dp3 instructions.

However, by simply using a color tint that gets blended with the original color we can get away with a single multiply. A color tint is less flexible, but we can improve it somewhat by making gray (0.5, 0.5, 0.5) the neutral basis, allowing us to add or remove strength from any of the three RGB components. We simply multiply the original color by double the tint value in the pixel shader.

output.Color.xyz *= input.TintTimesTwo;

In my current implementation, the tint value is calculated in the vertex shader. Each plant type has associated with it two tint values (this is specified at build time and set as shader parameters). Each plant instance has associated with it a minimum and maximum interpolation values. These are specified at runtime per plant instance (as shader variables, or instance data for instanced geometry).

For instance, a plant type may have tint values of (0.5, 0.5, 0.5) and (0.7, 0.3, 0.3). The former value is neutral while the latter value would add red and subtract blue and green. When rendering we interpolate between these two colors depending on the minimum and maximum interpolation values. For summer foliage, the interpolation values might be all 0. For trees that are turning color in autumn, we might use interpolation values of (0.5) and (1.0). That would mean we end up using a tint between (0.6, 0.4, 0.4) and  (0.7, 0.3, 0.3).

Which interpolation value is chosen between the minimum and maximum depends on a “phase” value that is associated with each leaf (or leaf entity). I use vertex colors for this. In fact, I can re-use the bending phase green vertex color that I use for the bending wind animation (see previous post). The result is shown below:

Per-leaf tints.

Again, the variation in tint values can be controlled per tree instance.

Ambient occlusion

A problem I had was that vegetation looked very flat under cloud cover. Since rain and clouds are going to be an essential part of my game, I need to do something about this.

In the Crytek paper about vegetation animation and shading, they mention using the alpha channel in their vertex colors for ambient occlusion.

I found a script for Blender that does a decent job of baking ambient occlusion into vertices:

Baking AO into vertices in Blender.

With this, we can cheaply “fake” some nice ambient shading.

Flat hemispheric ambient shading on the left. Vertex-based AO added on the right.

And with textures applied:

Flat hemispheric ambient shading on the left. Vertex-based AO added on the right.

I’m looking forward to seeing how this looks in game.

Color channel pre-processing.

Blender, the modelling program I’m using to draw the vertex colors used for wind animations (see last post), doesn’t allow you to draw to a specific component of a vertex channel. This makes it difficult to draw the red channel separately from the green and blue (and alpha which I’m now using for pre-baked ambient occlusion). So what I do instead is to create four separate vertex color channels:

I can draw this easily in Blender. Then I run a processing step in a content pipeline model processor that combines the four channels into one so the data is in a form I can use:

The vertex declaration that I end up using for models that participate in wind is the following:

Stream	Offset 	Type 		Method 	Usage 		UsageIndex

This is 32 bytes, which is optimal for the vertex cache. I could probably compress the normal and get more information in there if I need it. Notably, I don’t include tangent and binormal vectors that are needed for normal-mapping. I’ve found that normal mapping doesn’t improve things much with the size that I’ll end up drawing the models.


One comment on “Vegetation improvements

  1. Thanks for the tip regarding ‘* sign(vface);’ this is something I hadn’t gotten round to looking at for double sided polys. As always the style the article is written in makes it very pleasant to read.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

The Space Quest Historian

Let's Play's, Podcasts, and General Adventure Game Goodness

Harebrained Schemes

Developer's blog for IceFall Games

kosmonaut games

Development blog of "Bounty Road"


Turn up the rez!

bitsquid: development blog

Developer's blog for IceFall Games

Game Development by Sean

Developer's blog for IceFall Games

Lost Garden

Developer's blog for IceFall Games


Developer's blog for IceFall Games

Casey's Blog

Developer's blog for IceFall Games


Developer's blog for IceFall Games

Rendering Evolution

Developer's blog for IceFall Games

Simon schreibt.

Developer's blog for IceFall Games

Dev & Techno-phage

Do Computers Dream of Electric Developper?

- Woolfe -

Developer's blog for IceFall Games

Ferrara Fabio

Game & Application Developer, 3D Animator, Composer.

Clone of Duty: Stonehenge

First Person Shooter coming soon to the XBOX 360

Low Tide Productions

Games and other artsy stuff...


Just another WordPress.com site

Sipty's Writing

Take a look inside the mind of a game developer.

Jonas Kyratzes

Writer, game designer, filmmaker.

%d bloggers like this: