LayerMask in Unity

Couple weeks ago, I’ve posted a post about Bitfields for Multiple Flags. Recently I’ve found that LayerMask in Unity actually makes use of this trick. There’s a great discussions about LayerMask on Unity forum. Below is the forward of a great answer, which explain this feature clearly.


By Statement and Stelimar: http://answers.unity3d.com/questions/8715/how-do-i-use-layermasks.html

LayerMask, or any “Mask” in Computer Science, is an integer which uses its 32 bits to represent different flags. While it is an integer, the value doesn’t mean anything useful numerically. What matters is which flags are turned on or off.

In the case of LayerMask, each flag (each bit) represents one layer. If you select any game object from the editor you’ll notice the Layer field in the top right corner of the inspector. Clicking this will bring up a list of different layers. There are some predefined Layers, but you can also add layers. Notice how you have 32 slots for your layers. This is because Unity uses an integer type to store the flags in each of the 32 bits that make up an integer. As we can see, Layer 8 to 31 is available for us to define for our own game.

Since Masks use a single bit to represent if some option is true or false, a Mask can contain many different options. The same goes when thinking about Layers. A LayerMask specifies which Layers are enabled for any given mechanism. This enables us to tell the game that our GameObject “Don”, for example, is thought to belong to the user defined layer “NPC” and “Bob” to belong to the user defined layer “SpellCaster”.

Then, we make us of this information for example in a raycast call, specifying that the raycast should only return objects that belong to the “NPC” layer. An example when this could be used is when you want to target NPCs only with the click of the mouse. Then, imagine you have a game where you want to cast a spell, and that spell can only target Spell Casters. Then you’d only include the game objects that has the SpellCaster layer enabled. A third option would be to include both NPC and SpellCaster in the search. That way we would test both Don and Bob.

Finally some technical mumbo.

Technical


A LayerMask is represented as a 32bit integer. It also has a name table for naming the different bits so it is more readily understood by us game developers. Just to give some perspective, here’s how a 32 bit integer would look like if it was written.

0000 0000 0000 0000 0000 0000 0000 0000

I put a space between every 4th bit to make it easier to read.

The first bit is actually located on the right side. Think of this as being the first layer. Here’s how it would look if the first bit (or layer) would be set:

0000 0000 0000 0000 0000 0000 0000 0001

Since Unity predefines 8 layers, we can display the Mask for including all of those layers as such:

0000 0000 0000 0000 0000 0000 1111 1111

And if we like to search for NPC as from the example above, given we’ve put it in User Layer 8 (starting from 0, this is the 9th layer), we’d mask it as:

0000 0000 0000 0000 0000 0001 0000 0000 // NPC Layer

Just for completeness, this is how you should think a mask that includes NPC OR SpellCaster should look like, if we assume the SpellCaster was set to User Layer 9:

0000 0000 0000 0000 0000 0011 0000 0000 // NPC | SpellCaster Layer

Using bit shifting to create a layermask

The easiest way to produce a layermask is using the bit shift operator (<<). What this does is move each bit in an value to the left. So if you have a value of 6 (binary value of 00000110), and shift it two places to the left, you will be given a value of 24 (binary 00011000).

Remember that a value of 1 in decimal is 00000001 in binary. So if you bit shift 1 by n places to the left, you will end up with the nth bit being 1 and all other bits being 0. Based on this, the following code will produce a layermask which can be used to only test against the given layer:

var layer : int = 3;
var layermask : int = 1 << layer;

layermask will turn out to be 4 in decimal (00000100). Notice that the third bit is 1 (true).

Combining layermasks

To produce a layermask which will test against more than one layer, you can use the binary OR operator (|) to produce a new layermask. A bit in the new layermask is true if the corresponding bit was true in either one of the layermasks it was made from:

var layer1 : int = 3;
var layer2 : int = 5;
var layermask1 : int = 1 << layer1;
var layermask2 : int = 1 << layer2;
var finalmask : int = layermask1 | layermask2; // Or, (1 << layer1) | (1 << layer2)
// The bits look like this:
// (00000001) First bit is true in layermask1
// (00000100) Third bit is true in layermask2
// (00000101) Both first and third bit are true in finalmask

(Note that adding layermasks together with (+) is usually not wanted. For example, adding two layermasks that both contain the same layer, such as 00000011 (layer 1 and 2) and 00000110 (layer 2 and 3) would produce 00001001 (layer 1 and 4), while with the bitwise OR operator (|) you get 00000111 (layer 1, 2, and 3) as expected.)

Inverting a layermask

To test for all layers except the given layer, just use the bit inverse operator (~). The bit invert operator takes every bit in an integer and swaps it from 0 to 1 or vice versa. So ~00000010 equals 11111101, which would test for every layer except layer 2.

Note that if there are two layers you want to exclude, you need to combine the bitmasks with the OR operator before inverting:

var layer1 : int = 3;
var layer2 : int = 5;
var layermask : int = ~((1 << layer1) | (1 << layer2)) // NOT ~(1 << layer1) | ~(1 << layer2)

This would give you a value of 11101011.

Example


Allright, now that you see we’re setting bits to enable the different layers, let’s look at how we generate the bit field from scratch using C#. In my example, I will make three masks that can be used to query NPCs, SpellCasters, and “NPCs and SpellCasters”. We’ll usePhysics.Raycast in our example.

Example 1: Raycast vs NPCs

int NPCLayer = 8;
// This is used to illustrate which Layer we want to use.
int NPCMask = 1 << NPCLayer;
// This is used to generate the Mask derived from NPC Layer.
if Physics.Raycast(fromPosition, direction, Mathf.Infinity, NPCMask)
{
// We cast a ray onto a NPC!
}

Example 2: Raycast vs SpellCasters

int SpellCasterLayer = 9;
// This is used to illustrate which Layer we want to use.
int SpellCasterMask = 1 << SpellCasterLayer;
// This is used to generate the Mask derived from SpellCaster Layer.
if Physics.Raycast(fromPosition, direction, Mathf.Infinity, SpellCasterMask)
{
// We cast a ray onto a spell caster!
}

Example 3: Raycast vs NPCs and SpellCasters

int NPCLayer = 8;
int SpellCasterLayer = 9;
// This is used to illustrate which Layers we want to use.
int NPCMask = 1 << NPCLayer;
int SpellCasterMask = 1 << SpellCasterLayer;
// This is used to generate the Mask derived from the layers.
int CombinedMask = NPCMask | SpellCasterMask;
// We use the binary OR operator to combine masks.
if Physics.Raycast(fromPosition, direction, Mathf.Infinity, CombinedMask)
{
// We cast a ray onto a NPC or a SpellCaster!
}

Leave a Reply