ShiVa Multitouch explained – ShiVa Engine

ShiVa Multitouch explained

Ever since the iPhone revolutionized the smartphone world, multitouch has been a staple technology that allows users to interface with their mobile devices through simple taps and swiping gestures. Multitouch support has also been a part of ShiVa ever since version 1.6 many, many years ago. However, the behaviour of the touch handlers might surprise you if you have only ever worked with mouse pointers before. This tutorial will clear up some of the confusion as well as help you better understand and predict multitouch.

Inits and Limits

By default, multitouch is not enabled. All interactions on the touchscreen behave as if you were navigating and clicking with a computer mouse. Only one control element can be clicked at a time, which may be enough for a game like Flappy Bird, but is insufficient for a digital thumbstick + action buttons setup, since that one needs at least two touch points.
Before you can use multitouch, you must tell ShiVa to enable it for you. This is done simply with one line of code:

  1. input.enableMultiTouch ( this.getUser ( ), true )

In a multiplatform world though, you cannot predict which device your customers might use, so multitouch might not be supported. It is therefor better to set a control boolean (bEnableMultiTouch) and provide alternatives later on, like so:

  1. if input.enableMultiTouch ( this.getUser ( ), true ) then
  2. this.bEnableMultiTouch ( true )
  3. log.message ( "MT enabled" )
  4. else
  5. this.bEnableMultiTouch ( false )
  6. log.message ( "no MT - single touch/mouse only!” )
  7. end

You are now ready to use up to 5 fingers/touch points on the screen. This is the maximum number of taps ShiVa currently supports. Depending on your device, this might be more or less than what the hardware is capable of. Some screens and operating systems only support 2 touch points, others up to 10.

5 fingers on a hand/ler

ShiVa has 3 handlers that intercept touch input. Like all input handlers, those only work in the context of a user AIModel, not an object AIModel:

  1. onTouchSequenceBegin
  2. onTouchSequenceChange
  3. onTouchSequenceEnd

onTouchSequenceBegin is called when a finger touches an empty screen, and onTouchSequenceEnd is called when all fingers have stopped touching. Those two are not called when a second, third etc. finger touch or leave the screen, those kinds of changes are only detected by onTouchSequenceChange, the third and most interesting touch handler.
onTouchSequenceChange has a number of parameters that get refreshed every time the handler is called:

  1. onTouchSequenceChange ( nTaps0, nX0, nY0, nTaps1, nX1, nY1, nTaps2, nX2, nY2, nTaps3, nX3, nY3, nTaps4, nX4, nY4 )

nTaps0..nTaps4 are the 5 fingers that can be processed, with nTaps0 being finger one, while nX and nY denote their position on the screen.
nTaps are either 1 if the corresponding finger touches the glass, or -1 if not. Touches are always sorted: If there is only one finger interacting with the screen, nTaps0 is 1, but nTaps1..4 are -1. Likewise, if there are 3 fingers on the glass, nTaps0..2 are 1, but nTaps3 and nTaps4 are -1.
A word to the coordinate system used by nX and nY, as it is different to ShiVa’s HUD coordinates. We follow the industry convention of (0,0) being in the center of the screen, +1 Y being up, -1 Y being down, -1 X being left, +1 X being right:
touchCoords
In order to calculate ShiVa HUD coordinates (red system) from these values, you have to shift the value range +1 to get rid of negative values and then multiply by 50 to get values from 0..100:

  1. 50 * ( nX + 1 ), 50 * ( nY + 1 )

All in all, those are the 15 parameters to consider. You might want to let other scripts access those taps and positions, so it is a good idea to store everything in a member table. Initialize the table like so in your onInit handler, with tTouch being the storage table:

  1. table.reserve ( this.tTouch ( ), 15 )
  2. for i = 0, 14 do
  3. table.add ( this.tTouch ( ), nil )
  4. end

You can then store the tap and position info in onTouchSequenceChange through a table range call:

  1. table.setRangeAt ( this.tTouch ( ), 0, nTaps0, nX0, nY0, nTaps1, nX1, nY1, nTaps2, nX2, nY2, nTaps3, nX3, nY3, nTaps4, nX4, nY4 )

Access to these values can be done through a for-loop:

  1. for i = 0, 4 do
  2. local nTaps = table.getAt ( this.tTouch ( ), i * 3 )
  3. local nX = table.getAt ( this.tTouch ( ), i * 3 + 1 )
  4. local nY = table.getAt ( this.tTouch ( ), i * 3 + 2 )
  5. -- do stuff

So far, everything seems rather simple and predictable. Let’s try it out and see what happens!

Eeny, meeny

In the video below, white represents nTaps0, red nTaps1, and yellow nTaps2.

The behaviour shown in the video confuses most new developers at first. We will go through each case and explain why the colours change the way they do.
Single touchpoint: No surprises here. A tap (white) gets registered and updates nTaps0.
Two fingers: One finger is nTaps0 (white), the next one is nTaps1 (red). nTaps1 can be lifted at any time without any change to nTaps0. However if nTaps0 is lifted, then nTaps1 suddenly becomes nTaps0. You could now conclude that unused nTaps get eliminated and all subsequent nTaps get moved up the queue, but that would be wrong: When you touch another finger down, nTaps0 becomes nTaps1 again and the new finger gets the nTaps0 slot, not the other way round.
Three fingers: The issue becomes even more complicated once you add a third finger (yellow), although the pattern stays the same. Remove one of the lower nTaps and the higher nTaps get rearranged to fill the gaps, but once you put the _missing_ fingers down again, they assume a place in the previous order. Only now you cannot know anymore which finger was put down again, ShiVa will just assume the reverse order you lifted them, regardless of which actual finger it was.

Touching Tips

With all that new knowledge, it seems clear that you can never trust an nTaps to be the same finger from one onTouchSequenceChange event to the next since you can never know when the user lifts one finger while keeping others on the glass. Unlike keyboard keystrokes, you always need to process all nTaps at once, either through a long list of if-branches, or preferably a for-loop using a table as shown above.
You can also never be certain that the finger that has been removed is the same that was put down again. There are only two things you know for certain, IF a finger is on the glass (nTapsX == 1), and WHERE (nX, nY) it is. Fortunately, you also know where your HUD components are that you use for input. Consequently, for every onTouchSequenceChange event, you should check if the position of every nTapsX is within the area your control component. This can be done two ways: Either by component check or by coordinate comparison.
The component check is the easier solution, however it only returns the component that has the highest Z index. The function you need is

  1. hComponent = hud.getComponentAtPoint ( hUser, nPointX, nPointY )

Plug in your nX and nY for nPointX and nPointY and compare the resulting hComponent with a list of your active control components. If you want to use this method, it is recommended that you use a a separate HUD with clean or invisible label components at the highest possible indices which lie on top of everything else, merely marking the possible control areas.
Alternatively, you can check whether nX, nY lie within the area of an arbitrary component. This has the advantage of working with your existing HUD without the need for high Z indices or additional control HUDs. First, you need to determine the screen space coordinates of your control component using

  1. blx, bly = hud.getComponentScreenSpaceBottomLeftCorner( hComponent )
  2. trx, try = hud.getComponentScreenSpaceTopRightCorner( hComponent )

if your nTapsX position lies within these return numbers, then the tap is within the component:

  1. if ( nX < blx or nX > trx or nY < bly or nY > try ) then
  2. -- not within the area
  3. else
  4. -- nTaps has hit the component!
  5. end

This way, if a control area is hidden behind some other component like in this twinstick example, you still get the correct result.
touchTwinstick

Try it out

To get familiar with the nTaps behaviour, we have packed the test project from the video above into an STE. We highly recommend giving it a go on your iOS or Android device. Download touchtest_20150912.ste


  • slackBanner