Unity3D Coroutines and Problems With Them

If you are familiar with Unity3D, I’m sure you have used or at least heard about yield statements before. Normally, yield statements are not used in coroutines as there is no such thing as Coroutine in C# .NET. Instead, they are used to define iterators without having to create a class which implements IEnumerable/IEumerator or IEnumerable<T>/IEnumerator<T>. If you are not familiar with IEnumerator and IEnumerable interfaces or iterators, you can read the .NET Documentation, this StackOverflow thread and Yield Reference. Even though the yield statements are great to create iterators, they have some limitations:

  • You can’t pass any ref or out parameters to an iterator method.
  • Return type should be IEnumerator/IEnumerable or generic versions of these interfaces.
  • You can’t surround a yield statement with Try-Catch block, but you can write yield statement in Try-Finally block.

If you are using Unity Coroutines, you have to keep these restrictions in mind because the coroutine mechanism is implemented using iterators. As a result of this custom coroutine implementation, more items are added to the standard iterator limitations:

  • Coroutines run concurrently on single (main) thread.
  • Return value of a coroutine should be IEnumerator and it can yield null, YieldInstruction or CustomYieldInstruction.
  • If you are using nested coroutines,
    • When you stop the outer coroutine, it may not stop the inner coroutine. So you may have a dangling coroutine which causes problems during the runtime.
    • When you stop the inner coroutine, outer coroutine may not continue its execution from where it left off.

Concurrency

As you may already know, Unity Coroutines run concurrently. So in theory, you can do your heavy work such as AI calculations, WWW operations or other game logic without blocking the UI. But the Unity API is not thread safe, therefore coroutines are implemented to run concurrently in the same (main) thread. So, you can still access Unity API in a coroutine but if you do your heavy work without yielding, it will eventually block the UI and other operations.

Although they are working on the main thread, you can get help from them to start tasks in a new thread. You can write a tool that lets you to start a coroutine on background thread while simulating a coroutine internally. You can listen MoveNext calls from Unity and whenever coroutine yields to something, you can pause, resume or stop background thread by using an internal state machine. This way, you can switch from background to main thread whenever you want to access Unity API by yielding a certain command. There are also ready to use plugins to do this like Thread Ninja.

Returning a Value

Generally, you should not return a value from a coroutine unless it is nullYieldInstruction or CustomYieldInstruction. Even if you do, it will be regarded as null with a little performance loss. On each frame MoveNext is called for each coroutine, so yielding a value or null from a coroutine means that the coroutine has completed its work for the current frame and it can continue its execution on next frame. Therefore yielding a value or null means waiting for a frame. You can also yield other YieldInstruction implementations such as WaitForSeconds, WaitForEndOfFrame etc. or you can yield your own CustomYieldIntructor implementation.

The problem is, whatever the return value is, Unity engine receives it and you can’t see or use it.

private void Start()
{
    // Return type is Coroutine, so you can't access the return value here.
    Coroutine coroutine = StartCoroutine(TestCoroutine());
}

private IEnumerator TestCoroutine()
{
    // Try to return a custom value.
    yield return "value";
}

But there are some ways to receive values from coroutines.

Iterating Coroutine Manually

As I said before, Unity Coroutines are actually iterators. So you can iterate a coroutine manually and use its return value like this:

private void Start()
{
    // Return type is Coroutine, so you can't access the return value here.
    Coroutine coroutine = StartCoroutine(TestCoroutineRunner());
}

private IEnumerator TestCoroutineRunner()
{
    IEnumerator enumerator = TestCoroutine();
    while (enumerator.MoveNext())
    {
        // We can access the return value here.
        var value = enumerator.Current;
        yield return value;
    }
}

private IEnumerator TestCoroutine()
{
    // Try to return a custom value.
    yield return "value";
}

Using Nasty Callbacks

As we can’t pass ref or out parameters to the coroutines, we can pass a callback and wait for it to yield something:

private void Start()
{
    // Return type is Coroutine, so you can't access the return value here.
    Coroutine coroutine = StartCoroutine(TestCoroutine(OnNextvalue));
}

private void OnNextvalue(string value)
{
    // We can process the value in here.
}

private IEnumerator TestCoroutine(UnityAction&lt;string&gt; onNext)
{
    const string value = "value";

    // Use callback to inform caller that we are yielding a value.
    onNext("value");
    yield return value;
}

This solution might look innocent at first but as you get used to pass callbacks to coroutines, soon or later your code will become a mess. Even you won’t be able to follow the flow in your own code when you connect multiple coroutines to each other.

Using CustomYieldInsturcton

As of Unity 5.3, you can extend CustomYieldInstruction class to access return/result values. For example, in the example below we can access the Result value after the yield operation completes:

public class MyCustomYield : CustomYieldInstruction
{
    public override bool keepWaiting
    {
        get { return DoWork(); }
    }

    public string Result { get; set; }
    private bool DoWork()
    {
        // Do work for 2 seconds
        return Time.time <= 2;
    }
}

public class CoroutineTest : MonoBehaviour
{
    private IEnumerator Start()
    {
        var coroutine = A();
        yield return coroutine;

        Debug.Log(coroutine.Result);
    }

    private MyCustomYield A()
    {
        return new MyCustomYield();
    }
}

With this approach, you can create a class that extends CustomYieldInstruction, do your work and then access the custom result value. This lets you to write modular code for your custom coroutines but this doesn’t change the fact that you have to write a new class whenever you want to create a custom coroutine. In my next post, I’ll be talking about UniRx and how can we easily return values, filter and manipulate results without using extra classes.

Nested Coroutines

Have you ever tried to start or stop a coroutine in another coroutine? If the answer is yes, you’ve probably encountered strange problems. In Unity, coroutines are started using StartCoroutine method, and they can be stopped using yield break statement or StopCoroutine method. Wrong usage of these methods can cause problems while using nested coroutines.

Stopping The Outer Coroutine

Sometimes you need to start a coroutine in another coroutine, and when you want to stop them you naturally stop the outer coroutine and hope for inner coroutine to stop. Let’s consider the following example:

private IEnumerator Start()
{
    var coroutineA = StartCoroutine(A());

    yield return new WaitForSeconds(2f);

    StopCoroutine(coroutineA);
    Debug.Log("Stopped A");
}

private IEnumerator A()
{
    yield return StartCoroutine(B());
}

private IEnumerator B()
{
    while (true)
    {
        Debug.Log("Tick");
        yield return null;
    }
}

In this example, we start coroutine A when the game is started, and then it starts coroutine B. B immediately starts to print “Tick” to the console on each frame. And then we wait for 2 seconds and stop coroutine A by calling the StopCoroutine method by passing coroutine reference as argument. What do you think will happen when we stop coroutine A, do you think coroutine B will also stop?

The answer is No. Here is the output:

Ticking continues after A is stopped

It is really easy to believe that B will stop when we stop A if we don’t know how coroutines work. When we start coroutine A with StartCoroutine, Unity registers A to its internal coroutine list and calls MoveNext of it on each frame. The same is true for B, no matter who started it. So when you stop A with StopCoroutine, Unity does not remove B from its internal list and keeps calling MoveNext on B.

We can solve this problem in several ways. The first and ugly solution might be to move the code in B to A. But this might huge, monolithic coroutines which we don’t want to deal with. Another solution is starting and iterating B manually. We can do this in two ways:

private IEnumerator A()
{
    var enumerator = B();
    while (enumerator.MoveNext())
    {
        yield return enumerator.Current;
    }
}

or just (if you are using Unity 5.3 or later):

private IEnumerator A()
{
    yield return B();
}

The code above will not work if you are using an older version of Unity (older than 5.3). Previously, you were not able to start a coroutine without using StartCoroutine. Even though there is no documentation about this change in anywhere (changelogs, documentations etc.), this code should work without a problem. Now the output will be:

There are no ticks after the A is stopped

Stopping Inner Coroutine

Let’s consider the opposite of the example above, where we want to stop inner coroutine instead of the outer. Let’s have a look at this example:

public class CoroutineTest : MonoBehaviour
{
    private Coroutine _coroutineB;

    private IEnumerator Start()
    {
        StartCoroutine(A());

        yield return new WaitForSeconds(2f);

        StopCoroutine(_coroutineB);
        Debug.Log("Stopped A");
    }

    private IEnumerator A()
    {
        _coroutineB = StartCoroutine(B());
        yield return _coroutineB;

        Debug.Log("A continues...");
    }

    private IEnumerator B()
    {
        while (true)
        {
            Debug.Log("Tick");
            yield return null;
        }
    }
}

Again, we are starting coroutine A when the game starts and then it starts coroutine B. We wait for 2 seconds and stop coroutine B with StopCoroutine method. Do you think “A continues…” log will be printed?

The answer is again, No:

“A continues…” is never printed to console

Coroutine A is scheduled to run after coroutine B is finished, in other words when MoveNext of B returns false. There is a better explanation in here, but the problem is basically stopping coroutine B with StopCoroutine method. When we call StopCoroutine to stop B, Unity removes B from its internal coroutines list as soon as it yields to something. And to resume coroutine A, Unity waits for B to finish gracefully. This will never happen because we interrupted its execution before its MoveNext returns false properly.

Better Way To Stop Coroutines

Another (and elegant) way to stop the inner or outer coroutine is letting it stop gracefully by setting a flag or passing a cancellation token to the coroutine:

public class CancellationToken
{
    public bool IsCancellationRequested { get; set; }
}

public class CoroutineTest : MonoBehaviour
{
    private IEnumerator Start()
    {
        var cancellationToken = new CancellationToken();
        StartCoroutine(A(cancellationToken));

        yield return new WaitForSeconds(2f);

        cancellationToken.IsCancellationRequested = true;
        Debug.Log("Stopped A");
    }

    private IEnumerator A(CancellationToken cancellationToken)
    {
        try
        {
            yield return StartCoroutine(B(cancellationToken));
        }
        finally
        {
            Debug.Log("Finalize A...");
        }
    }

    private IEnumerator B(CancellationToken cancellationToken)
    {
        try
        {
            while (!cancellationToken.IsCancellationRequested)
            {
                Debug.Log("Tick");
                yield return null;
            }
        }
        finally
        {
            Debug.Log("Finalize B...");
        }
    }
}

Output will be like this:

Finalized Coroutines

Using this approach, we are letting the coroutine to finish gracefully. Most of the time, we want to finalize the coroutine, release the resources we use such as network sockets, disposable types etc. If we were using StopCoroutine to stop the coroutine, it was going to be interrupted so we could not see “Finalize A” and “Finalize B” logs.

Conclusion

Unity Coroutines may seem great for running concurrent tasks. The problem is, they have some limitations that you have to overcome, and sometimes you may have to write cryptic and ugly code to solve these problems. Also, since Unity’s source code is private, in some cases you may need to write a test code and see the results in order to understand what you will do while using coroutines. As an alternative to Unity Coroutines (or you can use with Unity Coroutines), you can use UniRx (Reactive Extensions for Unity) and overcome these limitations, and I’m planning to write my next post about it.

Implementing a virtual cursor in Unity

Hi everyone!

This is a technical post about how I changed the control mechanism of our phone interface in Paralycid; a horror and adventure game. You can find the demo project here.

I assume that you have knowledge in Unity and its UGUI system.

First of all, I will give you some information about our game. It is a fact that flashlights, lanterns, torches etc. are used a lot in horror games. Instead of using a flashlight or a lantern, our female protagonist carries her most valueable thing with her: a phone with various features such as dialing, messaging, navigation, flashlight etc. It’s like a real smartphone.

 

Phone in Paralycid

The phone has a very important place in the game as it is directly connected with the puzzles. Therefore, accessibility and ease of use is one of the most important things for us. For the “easy to use” part, we tried to control the phone with the arrow keys, “backspace” and “enter” keys on the keyboard. The player was using the arrow keys to navigate the menu, “backspace” to return and “enter” to select. However, this approach had a major disadvantage. The player had to switch from mouse to the keyboard whenever he/she wants to use the phone.

For those reasons, I’ve decided to try a virtual cursor for the phone and you can find the details of the process in this post. We are using Unity’s UGUI for the phone interface. All phone UI is inside a canvas which has a Screen Space – Camera render mode. Then we use a render texture to render the whole canvas on a quad inside a 3D phone object. So what we want was, creating a virtual cursor on this quad that can interact with all UI elements like the default cursor. Tl;dr: Here is the phone UI with new virtual cursor:

To do this, let’s create an empty scene first. Then create a Canvas with some elements (buttons, for example). Set Canvas’s render mode to Screen Space – Camera. Create a new camera, rename it to “CanvasCamera” and assing it as the camera of the Canvas. I’ve placed the buttons inside a Grid to refer a problem you may encounter:

Simple canvas

After creating the canvas and its camera, create a new render texture asset and assing it as the Target Texture of the CanvasCamera. Then create a quad object and assign render texture to quad. After these step, your scene should look like the image below. You can also use the demo project.

Quad with a render texture on it

At this step, you won’t be able to interact with the ui elements on the quad object. Let’s find out why. If you click to the EventSystem object, you will see that there are two components attached to this object: EventSystem and StandaloneInputModule. These are two main components/scripts that are responsible for the interaction in a scene. The EventSystem is responsible for processing and handling of events. And StandaloneInputModule is responsible for listening to input events coming from the mouse, keyboard, and touch screen. For information about input modules please read here.

Event System

When we move the default OS cursor and hover the quad, default input module (StandaloneInputModule) thinks that we are literally hovering the quad, it has no idea about the Canvas and its elements. Therefore, we should create a custom cursor in the Canvas and a custom input module which knows the exact positions of the cursor and where the Canvas elements are. Luckily, Unity UI is open source. So, we can modify and use UI scripts easily. But be careful, when you navigate through source code, the default branch is 5.2. Make sure that you are at the same branch as your Unity version.

First step is finding a sprite for the cursor and creating a cursor image in canvas. Then we need to move this image (cursor) with the mouse movements. For that, I’ve written a small script in VirtualCursor.cs. You can use the script file from the demo project. Adding this script to the cursor image will let you move it with your mouse. Then disable Raycast Target option of the cursor and set anchor to the left top as its edge will interact with the Canvas elements:

Cursor anchor and raycast target property

Then we will create our custom input module and an input manager which will let us switch between standalone and virtual input modules. We will have two gameobjects with EventSystem and an input module components and input manager will manage them by setting only one of them active in scene at a time. You can check them in the demo project.

VirtualInputModule will share most of its code with StandaloneInputModule and it will also extend Unity’s PointerInputModule. The difference will be using the position of our cursor image instead of using the position of default cursor while checking the cursor interactions. To do this, we need to download the StandaloneInputModule class at relevant branch from the Unity’s bitbucket page (it can be found under this path: UI/UnityEngine.UI/EventSystem/InputModules/), rename it as VirtualInputModule and change all input.mouseposition with the position of our custom cursor. PointerInputModule.GetMousePointerEventData() method also uses input.mouseposition so you may want to override this method with your own implementation thet uses the custom cursor position.

After setting the input manager and the custom input module, you can test the scene. When you press “C” key, cursor switches between virtual and the default one. You can find working codes in the demo project. If you notice, highlights will be problematic:

I’ve set the highlight color to red, and pressed color to blue.
If you notice, buttons are rarely blue although I rapidly click to them.

This is the result of locking cursor when enabling the virtual cursor. If you check the PointerInputModule, you can see lines like:

var targetGO = (Cursor.lockState == CursorLockMode.Locked ? null : pointerEvent.pointerCurrentRaycast.gameObject);

To fix this, we need to download PointerInputModule and alter these problematic lines, then use it as base class of the VirtualInputModule. Then the highlights will work properly.

And no, sadly, they will not.

Highlighting and pressing is working fine for the hovered buttons.
But as you can notice, other buttons are also highlighted.

For a Canvas with a single button in it, it will work perfectly. But if you are using multiple interactable elements in the Canvas (like I used within a grid), highlighting problems will arise again. This is caused by the Navigation property of the buttons. If you set the Navigation to none for each button, cursor will work properly:

Working virtual cursor in Canvas

You can find the demo project here. See you next time!