Translate

Saturday, August 23, 2014

How to render Parallax Background and Foreground Layers Completely in the Shader [Tutorial]

EDIT Oct 16 2014: Changed precision to lowp for fragment shader. Consider using this as a default for mobile, unless you really need something better.  I saw a 2x speed increase on Kindle by doing so.


Hi all!

Here's a quick little diddy for anyone curious on a quick way to get a layered effect in your 2D game (note: we are still using 3D vertices):

I did a little more progress on Hoarder Monkey  and I am happy to have parallax backgrounds working all in the shader and in 1 draw call per object type (static VBO, dynamic sprites, Semi-transparent VBO, semi-transparent sprites).  :)

The background scrolling is all handled in the shader, and works surprisingly well.  I also do some color tweaking to make things a little darker in the foreground and blue-ish in the background:


Notice the closest graphics are darker


Notice the background objects are a more blue/white hue.



In case your curious the shader code is pretty simple (as it should be for a shader):



Here's the Vertex shader for parallax.  Notice I just arbitrarily chose < -50 to be the furthest away, scrolling layer (right before the background).  This code will render based on camera movement half the distance that the action layer, will move (action layer == z distance where player is):

new_offset = u_camera_offset.xy / 2.0;

The closest palm trees and stuff  will move twice as fast as the action layer, abritrarily chosen to be distance Z > 50.0 (Keep in mind, in OpenGL -Z is further away from the camera, while +Z is towards the screen!):

new_offset = u_camera_offset.xy * 2.0;

uniform mat4 u_projection_matrix;
uniform float u_cutoff_alpha;
uniform vec2 u_camera_offset;
uniform float u_scale;

attribute vec4 a_Position;
attribute vec2 a_texture_coords;

varying vec2 v_texture_coords;
varying float v_cutoff_alpha;
varying float v_zdepth;

void main()
{

  vec2 new_offset = u_camera_offset;
  v_texture_coords = a_texture_coords;

  if(a_Position.z < -50.0)
  {
    new_offset = u_camera_offset.xy / 2.0;

  }
  else if(a_Position.z > 50.0)
  {
    new_offset = u_camera_offset.xy * 2.0;

  }

  vec4 pos =  u_scale* (vec4(new_offset.x, new_offset.y, 0.0, 0.0) + 
      vec4(a_Position.x, a_Position.y, a_Position.z/2.0, a_Position.a));

  gl_Position = u_projection_matrix * pos;
  v_cutoff_alpha = u_cutoff_alpha;
  v_zdepth = a_Position.z;

}


Ok, now that scrolling business has been taken care of, for that oh so nice cartoon layered depth illusion.  Now what about coloring.  How about we color the background layers a little bit blue-er and white than the closer layers, and maybe the foreground (closest branches and such) can be a little more black?

// The Blue/white far scrolling layer
orig_color.b *= 1.90; 
gl_FragColor = mix(vec4(1.0,1.0,1.0,orig_color.a), orig_color, .9);

// The closest layer (more black color)
gl_FragColor = mix(vec4(0.0,0.0,0.0,orig_color.a), orig_color, .5);

// The normal action layer (where player is etc, no color changes):
gl_FragColor = orig_color;


#ifdef GL_ES
precision lowp float;
#endif

uniform sampler2D u_texture;
varying vec2 v_texture_coords;
varying float v_cutoff_alpha;
varying float v_zdepth;

void main()
{

  vec4 orig_color =  texture2D(u_texture, v_texture_coords);

  if(orig_color.a < v_cutoff_alpha)
  {
    discard;
  }
  
  if(v_zdepth < -50.0)
  {

    orig_color.b *= 1.90;
    gl_FragColor = mix(vec4(1.0,1.0,1.0,orig_color.a), orig_color, .9);

  }
  else if(v_zdepth > 50.0)
  {

    gl_FragColor = mix(vec4(0.0,0.0,0.0,orig_color.a), orig_color, .5);

  }
  else
  {
    gl_FragColor = orig_color;

  }
  
}


So there you have it.  In a nutshell this is how you can do parallax scrolling backgrounds all in the shader.  Of course you will still want to order your draw polygons to draw first:  closest to further away, and for semi-transparents draw first: further away to closest.

Draw ALL your semi-transparents last. Even after your background!  If you don't you'll get fun artifacts and it won't blend correctly.  Also, don't forget to use a TextureAtlas to reduce all your draws to just a few, and VBOs for any non-moving game objects (background decorations, non-moving platforms etc).

Happy GameDev-ing!

Sunday, August 17, 2014

How To Set Up Your Game Event Queue (Threaded) [Tutorial]


Hi all,

Finally back with a simple, but hopefully helpful, tutorial for you all to get an working event queue in a threaded environment.  In this case I will be using Android C++ and JNI as my playground to demonstrate.

What is an event queue


An event queue is a list of events that has occurred since your last game loop iteration.  Possible Events are:

  • Button presses on game controllers
  • Screen touch events (press up/down, multi touch, etc).
  • Keyboard events

So what is the point of this anyhow?  


Many times events will occur in the middle of your game loop, asynchronously.  In Android, they occur on a different thread by default, usually called the "UI thread".

If you process them as they come in on your "UI Thread" you could end up with some nasty concurrency bugs and crashes due to processing the event (in your game etc) right while the game loop thread is updating the same data.

This leads to some epic head scratching and lovely stack traces and pulling out some addr2line kung fu! (or ndk-stack in Android > NDK r6)

08-22 23:27:40.730: INFO/DEBUG(65): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
08-22 23:27:40.730: INFO/DEBUG(65): Build fingerprint: 'htc_wwe/htc_bravo/bravo/bravo:2.2/FRF91/218634:user/release-keys'
08-22 23:27:40.730: INFO/DEBUG(65): pid: 2474, tid: 2485  >>> com.test <<<
08-22 23:27:40.730: INFO/DEBUG(65): signal 11 (SIGSEGV), fault addr 00000001
...
08-22 23:27:40.790: INFO/DEBUG(65):          #00  pc 00018656  /data/data/com.t
...

Let's spare you all that. You've got enough to deal with in finishing your game!

Android only solution


Ok, if you're only going to be using android this could be the easiest solution.

// In the class that handles your input
  
  //*****************************************************************************
  /**
   *  Handle all touch events for the game
   */

  @Override 
  public boolean onTouch(View v, MotionEvent event)
  {

    final View v1 = v;
    final MotionEvent event1 = event;
        gl_surface.queueEvent(new Runnable() { 
            public void run()
            {
              onTouchRenderThread(v1, event1);
            } });
    return true;
  }

// ....


Take note of queueEvent.  queueEvent will queue the event handler you've specified (in this case onTouchRenderThread()) to be called on the Render Thread.  If you're lazy like me, you probably do all your game loop within the render thread, which is fine for small simple games.  This works great and pipes the event to be consumed by the render thread and you don't get any concurrency issues and all is right with the world.

Portable solution




Ok, what about if we want to port to IOS, Windows phone etc and we don't want to rewrite the event handling code?  Maybe these platforms don't have the same convenience function(s)?  That's where rolling your own event queue could make sense for your project.

We could write the solution in pure C++ (well except we are using JNI for capturing the event here but ignore that :) ), that way the crux of the event code is in a portable format that can be logic tested easier on multiple platforms.

If using Java for handling in coming events in Android, you'll still have this portion of (not as portable) code.  Notice there's no need for queueEvent, as we are going to handle concurrency by just mutexing the read and writes to the event queue itself.

Put on your C++ hat!  Here's a simple event queue for us to use:

/**
 *  Input types for the input queue
 */
typedef enum {
  MOVE_SINGLE_FINGER_INPUT_EVENT,
  TOUCH_DOWN_SINGLE_FINGER_INPUT_EVENT,
  TILT_INPUT_EVENT,
  TOUCH_DOWN_MULTI_FINGER_INPUT_EVENT,
  TOUCH_UP_INPUT_EVENT,
  PINCH_ZOOM_INPUT_EVENT,
  SCROLL_INPUT_EVENT,
}InputEventType;

//*****************************************************************************
/** 
 * Describe an input event
 */
class InputEvent
{

public:
  InputEventType type;
  int finger_num;
  float x;
  float y;
  float z;
  double percentage_zoom;

  InputEvent(InputEventType type_, int finger_num_, float x_,
  float y_, float z_, double percentage_zoom_):
    type(type_),
    finger_num(finger_num_),
    x(x_),
    y(y_),
    z(z_),
    percentage_zoom(percentage_zoom_)
  {
  }

};

//*****************************************************************************
class InputHandler
{

  private:

   //...

    /// Mutext to make sure input isn't written to by two threads at the
    /// same time
    pthread_mutex_t mutex;

    /// List of events to consume on render thread when ready
    std::vector<InputEvent> queued_events;

//...
    void queueEvent(const InputEvent& event);
    void getQueuedEvents(std::vector<InputEvent>& events);

    void processTouchDownDetected(int finger_num, float x, float y);
    void processTouchDownSingleFingerScreenSpace( 
        int finger_num, int x, int y);
    void processTouchDownMultiFingerScreenSpace(
    int finger_num, int x, int y);

    void processTouchUpScreenSpace(int finger_num, int x, int y);
    void processMoveSingleFingerScreenSpace(int finger_num, int x, int y);

//...

    InputHandler():
      first_touch(),
      up_touch(),
      paused(false),
      fingers_down(0)
    {

      pthread_mutex_init(&mutex, NULL);
      memset(last_touch, 0, sizeof(last_touch[0])*INPUT_FINGERS_SUPPORTED);
    }

    InputHandler(const InputHandler& i);

  public:
    static InputHandler& getInstance()
    {
      static InputHandler input;
      return input;

    }
//...

}
I put in some basic events that you can use for touch, scroll and zoom etc.  If you'd like to extend it for controllers/keyboards you can add int input_button to the InputEvent class etc and another event to the enum KEYBOARD_INPUT_EVENT, etc.

Here's that same Java code from earlier without the queueEvent function, shortcut. We don't need it since our own C++ InputQueue code will handle everything.

// In the class that handles your input

  //*****************************************************************************
  /**
   *  Handle all touch events for the game
   */

  @Override 
  public boolean onTouch(View v, MotionEvent event)
  {
    onTouchRenderThread(v, event);
    return true;
  }

// ....

This is the Java Touch event handler (all variants of touch events). It just makes sense of the types of touch events.

 What this does in this section isn't so important (so don't get bogged down by it), it's just shown as an example of how the data flows to native event handling.  I just wanted to be a bit verbose:

  //*****************************************************************************
  /**
   *  Handle all touch events for the game (to be called in render thread)
   */

  public boolean onTouchRenderThread(final View v, final MotionEvent event)
  {

    // Send all point data
    int touches = event.getPointerCount();
    int action = event.getActionMasked();

    //Log.v(Logging.TAG, "OnTouch() called..\n");

    switch(action & MotionEvent.ACTION_MASK)
    {

      // First finger down
      case MotionEvent.ACTION_DOWN:
        {

          active_pointer_id = event.getPointerId(0);

          processTouchDownDetected(touches-1, event.getX(), event.getY());
          break;
        }

        // First finger up
      case MotionEvent.ACTION_UP:
        {

          active_pointer_id = event.getPointerId(0);

          processTouchUpDetected(touches-1, event.getX(), event.getY());
          two_fingers_were_down = false;
          break;
        }

        // Second finger down
      case MotionEvent.ACTION_POINTER_DOWN:
        {

          Log.v(Logging.TAG, touches + " ACTION_POINTER down...\n");
          processTouchDownDetected(touches-1, event.getX(touches-1),
              event.getY(touches-1));

          two_fingers_were_down = true;

          //          Log.v(Logging.TAG, "pointer[0] " + last_touch[0] +
          //              "pointer[1]" + last_touch[1]);
          //          processMoveSingleFinger(0, event.getX(0), event.getY(0));

          break;

        }

        // Second finger up
      case MotionEvent.ACTION_POINTER_UP:
        {

          Log.v(Logging.TAG, touches + " Finger up...\n");

          break;
        }

        // Moving finger down
      case MotionEvent.ACTION_MOVE:
        {

          //final int index = event.findPointerIndex(active_pointer_id);
          if(touches > 1)
          {

            processTouchDownDetected(0, event.getX(0),
                event.getY(0));
            processTouchDownDetected(1, event.getX(1),
                event.getY(1));

            double distance = last_touch[0].distance(last_touch[1]);

            processPinchZoom(distance);
          }
          else if(two_fingers_were_down == false)
          {
            Log.v(Logging.TAG, "Moved Finger down...\n");
            processMoveSingleFinger(0, event.getX(0), event.getY(0));
          }

          break;
        }

    }

    if(touches <= 1 || 
      (action & MotionEvent.ACTION_MASK) != MotionEvent.ACTION_MOVE)
    {

      last_distance = 0;
    }

    return true;


  }
// ....


  /// Input event functions
  private native void processMoveSingleFinger(int finger, float x, float y);
  private native void processTouchDown(int finger, float x, float y);
  private native void processTilt(float x, float y, float z);
  private native void processTouchDownMultiFinger(int finger, float x, float y);
  private native void processTouchUp(int finger, float x, float y);
  private native void processPinchZoomNative(double distance);
  private native void processScroll(float x, float y);


Ok back to C++.  The following is an example of a Touch input handler in C++.

The oddly named function Java_com_razzlegames_HoarderMonkey_MyRenderer_processTouchDown is where JNI sends this particular touch event.

The function  InputHandler::processTouchDownSingleFingerScreenSpace, is where the event, is finally queued for the render thread (or whatever your main game logic sits in) to consume.

Keep in mind, we are still 100% in the UI thread for these call backs.

  

//***************************************************************************
/**
 *  Call back for single finger touches (in screen space)
 */

void InputHandler::processTouchDownSingleFingerScreenSpace(
    int finger_num, int x, int y)
{

  fingers_down++;
  if(paused)
  {
    return;
  }


  first_touch = Vector2Di(x,y);
  Vector2Df world_coord =
    InputHandler::transformPointScreenToWorld(Vector2Di(x,y));

  setLastTouch(x, y);

  queueEvent(InputEvent(TOUCH_DOWN_SINGLE_FINGER_INPUT_EVENT, finger_num, 
        world_coord.x, world_coord.y, 0, 0));

}


// This is just so we can decide not include this for other build platforms!
#ifdef ANDROID_NDK   


//...

//*****************************************************************************
/**
 *  Call back for single finger touches
 */

JNIEXPORT void JNICALL Java_com_razzlegames_HoarderMonkey_MyRenderer_processTouchDown(
    JNIEnv* env, jobject obj, jint finger, jfloat x, jfloat y)
{

  LOGD("Process touch down!");
  InputHandler& input_handler = InputHandler::getInstance();
  input_handler.processTouchDownSingleFingerScreenSpace(finger, x, y);

}

//...

#endif // ANDROID_NDK

Ok here's where we are finally going to queue up the message for main game loop thread to consume:

  
//*********************************************************************
/**
 */

void InputHandler::queueEvent(const InputEvent& event)
{
  pthread_mutex_lock(&mutex);
  queued_events.push_back(event);
  pthread_mutex_unlock(&mutex);

}

This is where we get the events.  Call this from the main Game loop thread:

//*********************************************************************
/**
 */

void InputHandler::getQueuedEvents(std::vector<InputEvent>& events)
{

  pthread_mutex_lock(&mutex);
  events = queued_events;
  queued_events.clear();
  pthread_mutex_unlock(&mutex);

}

 [ I decided to use a pass by reference, std::vector<InputEvent>& events, since I just didn't want to waste recreating all the vector and all the events on the stack by passing it back with return. If you prefer that method, it probably won't be much of a concern for performance. On average you won't queue up that many user input events in 1/60th - 1/30th of a second :)  (from my tests I was able to get a backlog of events of only a max of 2 or 3).  ]

Let's put it all together...


This is where all your hard work pays off and you actually use the event queue from your main Game loop, thread.

It's completely thread safe now since we are using pthread mutex to block reading and writing to the queue.

//*****************************************************************************
/**
 * Advance the game simulation
 */

void Game::stepGame()
{

  Renderer&amp; renderer = Renderer::getInstance();
  renderer.drawFrame();

  processInputQueue();

//.....

}


//*****************************************************************************
/**
 * Process all Input events from the input thread
 */

void Game::processInputQueue()
{

  InputHandler&amp; input_handler = InputHandler::getInstance();
  vector<InputEvent> events; 
  input_handler.getQueuedEvents(events);
  for(InputEvent e: events)
  {
    // Do things with events!
  }
}


Yay, you have synchronized Input Events!




You should now have a working structure to build your InputEvent queue for Android and tie it to your C++ game logic.  You can use this same InputEvent queue code for IOS, Android, Linux, probably even Windows with a Cygwin compiler.  

Please feel free to ask any questions below and have fun coding and learning!