When reading current video game publications, one sees a persistent misconception concerning the term “Cel Shading” and its use in video game graphics.
For all intents and purposes, Cel Shading as a term is currently being used as a misnomer. The term itself comes from traditional animation, where objects and characters were painted on transparent cels (short for celluloid). As a shading technique in computer graphics, it describes a method to emulate a non-realistic aesthetic similar to hand-drawn flat shadowing.
Traditionally, shading refers to the process of adding depth to a picture by the use of shadows and light, specifically in computer graphics. This refers to having a set of mathematical descriptions for the behavior of materials, shadow and light sources.
3D Computer Graphics Imaging is a different beast to traditional art. In the latter, the artist has direct control over the process of shading and flow of light. In the former, a set of intricate rules describes the general behavior of a material and the computer renders the image based on those values. With CGI it is uniquely difficult to emulate traditional artistic expression because shading is handled mechanically by mathematical algorithms and will produce consistent, realistic, results.
Most lighting and shader models are based on real world physics and are concerned with realistically replicating light in the virtual space and they do so with precision and accuracy. The trouble is that traditional art is in many cases not interested in replicating the real world but rather in representing a simplified version of it.
It is hard to tell a machine to “do it wrong.”
The above image shows three spheres generated in the popular free 3D program Blender. From left to right the light source does not change, it’s a static directional caster with a set intensity and distance, nor does the texture or model for that matter, yet the difference in display is quite significant by only manipulating the rules for shadow and light reception of these materials.
This is the power of shading.
The furthermost right sphere is shaded with what is generally accepted as “cel” or “toon” shading. If you observe carefully, you will notice that both shadow and light drawn onto the material have hard edges and do not feature a soft gradient like in the other examples.
This technique is was often used in traditional animation because flat, crisp shadows are faster to draw by hand than colored gradient shading. When you look at traditional animation classics you will notice that a lot of them employ elaborately painted, static backgrounds but use simple shading techniques and flat color for their animated characters or foreground elements.
Another element of traditional animation and comic art is the so-called “line-art” of an image – an outline around the character or element that differentiates it from other objects in the scene. Additionally, in black and white drawings, the line-art serves to accentuate certain characteristics of material texture, like cloth. As we observe in the following example, outlines or visible line-art is not prominently featured in paintings because the differentiation happens through color, directional brush strokes and other specific shading techniques that use tonal changes to convey depth and object definition.
Compare these two images from All-Star Superman (2011). On the right we have a still frame from the movie and on the left we have the cover image for the DVD, both images show the same scene.
The left image is what we would consider painted and the right image is what we would consider drawn, the differences are missing line-art in the left image and a higher fidelity of shading as well as a distinct texture given to the image by the use of a brush (even if a digital representation of one.)
Line-art itself can take many different forms depending on the tool being used. There is a clear distinction between a ball-pen and an ink-pen, distinguishing stroke width and direction, and most video game graphics trying to emulate line-art have trouble with this concept as it is hard for a machine to simulate the hand of an artist.
The internal rules for line drawings are a bit different from painting, most notably the “physics” of how distance and depth work with a clear, crisp outline around objects. Usually depth and distance is conveyed by line strength and definition. Modern, non-traditional effects like depth of field or blur are usually a bad choice for drawing; they strongly jar against the crispness of an illustration.
Many video games mix cel shading and modern post-processing effects, ending up with an incoherent mish-mash that is neither truly cel shaded nor photorealistic. Take for example Borderlands, and its artificial line-width which is independent of distance and object, in comparison to Okami in the image below.
In the case of Borderlands this is a problem with how graphics engines tend to handle the effect of outlines, and how it is composited with their attempt at emulating hatching with their textures. Usually when generating 3D model outlines the engine creates a set of back faces from the mesh that are then drawn black, then the model is rendered normally and both images composited. This means that every 3d model gets an outline that is equal in thickness, no matter the distance or size. This leads to visual inconsistency with the fixed hatching technique used for textures because the textures are not able to modify themselves and present a consistent line-width.
I am not familiar with the technology that Okami uses, but you can see that the static texture line-width is more consistent with a drawn art style, by manipulating the outline width as well as simulating stroke direction.
The trap that many writers in the video game industry fall into is associating outlines or drawn textures with cel shading. Cel shading is a shading technique and not relevant to coloring or inking.
Merely employing an outline does not qualify as cel shading. In the above example we have multiple models with and without outlines, but only two of them are actually cel shaded. The furthermost top, right example is a special sketch or ink shader I developed for architectural illustration. It is entirely light-based and does not rely on outlines or textures. Furthermore, it reacts to distance with stroke-length and thickness being dynamically generated.
The Legend Of Zelda: Wind Waker, for example, doesn’t feature any outline around models, and is the typical example of cel shading.
It should be noted here that even the (contested) wikipedia article on cel shading falls into the trap of using example-games that do not feature cel shading (Borderlands, Crackdown). Over the years the term has mutated and shifted in meaning through use by ignorant writers in the media, and today we have a situation where some games are described as having cel shading, even though they neither feature cel shading, nor silhouette outlines but rely on texturing as the source for their painted aesthetic.
A positive example would be the recent Guilty Gear Xrd trailer showing a fully 3D rendered environment but with a typical cel shaded anime aesthetic. When observing the motion of movement as well as the techniques translated into 3D, it becomes apparent that a lot of effort and study went into staying true to the 2D aesthetic. Sword slashes for example feature a typical 2D blur effect of extended color or speed-lines, instead of a modern post-production effect like motion blur.
When trying to emulate a comic book style, or generally a drawn aesthetic, the designer must understand the internal logic of traditional art and the techniques associated with it. Hand drawn animation for example rarely relies on motion blur and reflective materials are treated with a modicum of simplification in illustration. Drawn visuals simply work differently from the realistic mathematical models employed by current graphics engines.
Simply adding outlines or processing your textures with the paint daubs filter isn’t enough to convey a drawn aesthetic.