If I were writing an app.....

Discussion in 'Wii - Emulation and Homebrew' started by KiiWii, May 11, 2009.

  1. KiiWii

    KiiWii GBAtemp Guru

    Nov 17, 2008
    United Kingdom
    Basically I've started making something, that will recognize certain specific gestures made by the wii remote or input device.

    Does any one know how I could get the screen to recognize a circle having been drawn?

    Imagine for example: using a basic drawing tool > you draw a rough circle with the pen > and wanted the computer to print to screen "a circle has been drawn" (as opposed to if you drew a line, it would state that a line had been drawn)

    I cant get my head around the process I would need to go through to varify that the rough shape of a circle had been drawn/gestured, and then perform an action...

    .. been trying various ways for hrs now and none are working anywhere near usefully....

    Any ideas, or even better pseudo-code, appreciated


    (ps sorry if this is vague and/or in the wrong section)
  2. ddp127

    ddp127 GBAtemp Fan

    Jan 14, 2009
    i dont know how difficult this is, but try something like this:
    1. change the background color from 00,00,00 to 00,00,01 or something like that (in the same way as the fill option in paint)
    2. if there is a circle, there will be a piece that is still 00,00,00, if not, everything will be 00,00,01
    3. change the color back, and check again, and start over from step 1
    4. to make sure it is a circle, you also need to check if the end and the beginning of the drawn line touch eachother...

    ( sorry if this also is a little vague)
  3. Jacobeian

    Jacobeian GBAtemp Advanced Maniac

    May 15, 2008
    gesture recognition is generally done by game developper using specific libraries
    even basic gesture recognition can be very difficult to achieve and you need to developp advanced algorithms for that, using not only the wiimote raw position in space but also the gravity values returned by accelerometers

    what you describes is more using the wiimote position as a pencil, this can easily be done by converting the acquired wiimote position (using WPAD_IR function) into a screen dot position
  4. FAST6191

    FAST6191 Techromancer

    pip Reporter
    Nov 21, 2005
    United Kingdom
    +1 to the others, this is possibly one of the hardest problems in computing. For one person it is easy enough (especially if you "train" them) but for a "general" population you are facing serious problems. Firstly I am not overly familiar with the wiimote's limits from a programmers perspective so you will need to read up on those before you get into things.

    First read up on fuzzy logic (a good start http://www.seattlerobotics.org/Encoder/mar...uz/flindex.html ), it is about the only way to account for the inherent unpredictability of the input (the person and the device: people are not so hot at gestures and the wiimote is not known for greatness in this regard either).

    In the simplest form if you have a preset range of tasks then assigning simple movements (move to the right for a circle, left for a square, up for a triangle.....) is probably the easiest option. I assume this is not the case though so you will have to account for speed, range of movement (most people can not even draw a freehand circle of a decent size with full concentration let alone using a wiimote while otherwise occupied).

    The wiimote is a crude 3d space sensor so assign values at or above limits/scales of sensitivity for each "axis". I will leave it to you to work out what range you want to cover, it might also be worth enhancing sensitivity between certain ranges (most people will use either wrist rotation or movement about the shoulder and elbow which gives you about 0.8m for comfortable movement, if you are going to have them move on their feet then things change again).
    As mentioned you also have accelerometers, this can save you some minor effort and decrease the "refresh rate"/polling interval if you go that router.

    After this I would start the other side of things and get as many people as you can to draw shapes/perform movements to try and get some idea for the "fuzziness" required. You can not really do this yourself as you run the risk of training yourself and trying to correct things/adapt for your code.

    A final though before psuedocode, many problems for gestures in games arise from somewhat harsh penalties for "failing" a gesture. Perhaps a "slowdown" of the game upon failure to allow for a second go.

    psuedocode: "#"is start of a comment line, "[" is what I will put when you need to fill in the blanks, my pseudocode will likely be a hybrid of different languages but hopefully you can follow it. I make little to no attempt to make a clean running program/library.
    Nothing that comes should be "new" but none the less I stake no claim on what follows.

    #Wii Gesture recognition

    #y z
    #| /
    #that will be my definition of the axes, you may want to change that to a more conventional (from a 3d programming perspective) method.
    #I will try to stick to those axes and not define a second set, depending on your app though you might like to but it should not be that difficult to change.

    Set poll interval = 60 milliseconds #this is about 17Hz, another option is interrupt based rather than polling based. You will have to set a range for an interrupt though which tends to get a bit hairy to do from a speed perspective (especially on hardware like that of the wii).
    #My thought would be that interrupts are better for games while polling based is better for applications.

    Set X = quad #x- variable# Hopefully 32 bits will provide a fine enough resolution.
    Set Y = quad #y- variable# A secondary idea might be to use a finer resolution in the "typical range" and then reduce the effectiveness on the outside range
    Set Z = quad #z- variable# Example: in the "usual" 80 cm sided box give 90cm to the high precision but then only add a bit for movements twice as far outside of it.
    # People dislike calibrating things so it might be worth trying some form of "learning" to define the "typical range".
    #I will leave it to you to choose if you want signed variables, I am inclined to leave it and set the bottom left of the person as the origin going away, up or "right" as positive and then calculate after the fact.
    #If it has the provisions in hardware then by all means use them.

    Set xaccel = signed float #you want signed here here to simplify the maths and float should also help.
    Set yaccel = signed float
    Set zaccel = signed float

    #You also have rotational sensors, without testing I am not inclined to say but I suspect rotation plays a role in the given movements (how many people twist their wrist to draw a circle? now for a similar size square what happens?).
    #I seriously doubt rotation of more that 180 degrees about any axis will occur. I suggest radians as we are ultimately going to be dealing with high level geometry and probably calculus using it and degrees are not especially suited for either.
    #I will leave it up to you to choose whether to "retain" until calculation or use data during calculation and incur the accuracy penalties (remember errors stack/compund and fuzzy logic is somewhat long winded).
    Set xrot = signed float
    Set yrot = signed float
    Set zrot = signed float

    #from here my instinct would be to try to make branching code (potentially slower although able to be done in real time) although you could make "after the fact analysis" if you wanted.
    #A further option is some kind of "learning algorithm" but this takes us into very complex territory.
    #This is also where the fuzzy logic comes in, start with the geometric shapes (circle being radius squared= x squared + y squared in the simplest form).
    #Your main problem here is 1) starting point (do you start the circle at a given "compass point", any "compass point" or some arbitrary point.
    #This is not so much of a problem for simple shapes but as a thought exercise try it for a semicircle and all the places it might start.
    #Personally I suggest breaking down a shape into sections: squares into lengths/sides, circles into quarters, triangles into subtype and then sides......
    #Another problem is over or under rotation, even with fuzziness you are likely to find yourself outside a strict set of parameters. Interrupts should take care of this but you could also try disregarding data.
    #Matrices are probably the easiest way of doing things here. You will also want to account for sizes, the easiest way to do this is to differentiate or interpolate.
    # You could also try normalising or even reducing the value contained to a simple term (left movement=0, right=1).
    #Whether you pile everything into one large matrix or use many is up to you, I would lean towards the halfway house at first of using several bigger matrices.
    #Obviously the first step is to make the matrices.
    #For polling:
    #nicely you have 9 main variables which is a 3x3 matrix although you will need to write the pulse number somewhere else (probably a simple adder will do).
    on_event(pulse) write
    (x, y, z, xaccel, yaccel, zaccel, xrot, yrot, zrot) to mem location.

    #At 17Hz you are looking at about 6 kilobytes for a 10 second gesture/recording time. This will go up if you account for boundaries but probably not by much.
    #Interrupts could be useful for determining end of gestures.

    #For interrupts, this one gets trickier as you will have to decide what you want to use, probably on a per game or even per action basis.
    #Generally you will want a rest value (this need not be “zero” or even a set value:
    #think sweeping action: acceleration changes and will probably be in a sinusoidal motion,
    #As a nod to the fuzzy logic that governs this thought exercise you could even use multiple interrupts to trigger an event: one interrupt primes another.
    write (x, y, z, xaccel, yaccel, zaccel, xrot, yrot, zrot) to mem location (a) # this is your base value.
    Set interrupt (x-xa > 30)# this could be anything.
    write (x, y, z, xaccel, yaccel, zaccel, xrot, yrot, zrot) to mem location (?)

    #either way we end up with a bunch of matrices
    #The nice thing about matrices is that you can kill various terms quite simply and manipulate others.
    #here you would perform various operations on the matrices depending on what you needed.
    [operations as required.

    write “matrixmod(number)” to memory location (?)

    #You could now do interesting things. On the one hand you could normalise a matrix (set a y value to 0 if it falls between a given range for example) and then work with “precise” maths or you could use the various range based instructions of a processor or features of a language.

    #Back to branching. The easiest way here would be to have a library of shapes/gestures/sections as required and then only compare key things.
    #This will of course up memory demands but I suspect a form of dynamic generation could be used (shapes easily being able to be generated from algorithms).
    #Such a section would likely read:

    compare matrixmod(number) to reference matrix(othernumber)# you need not use a matrix either, you could use a single term.
    matrixmod(number)=referencematrix() then goto routine “circle?”
    compare matrixmod(number) to reference matrix(othernumber+1)#this might be a good time to start a new routine.

    #So far we have only flirted with fuzzy logic, in practice you may completely disregard a data set (I suspect this will mainly be the rotation)
    # and more interestingly you may bring it in and take it out as necessary; not sure if the square is a square where you were in the previous section: bring back rotation.
    [here you put the "end game" of determining what the gesture is.
    [I suggest you make this a library and simply write a value to the memory and then have said memory referenced by something else.
    End of psuedocode.

    Good luck writing the apps.
  5. Sterling

    Sterling GBAtemp's Silver Hero

    Jan 22, 2009
    United States
    dang FAST that is one mean block of text [​IMG] it's beautiful Z.Z Actually from what i read i gather you know your c or c++ or both... correct me if I'm wrong. This Will definatly help this guy [​IMG] (if he can keep up the good work and not get dicouraged)

    Also KiiWii keep up the good work [​IMG]
  6. KiiWii

    KiiWii GBAtemp Guru

    Nov 17, 2008
    United Kingdom
    Been very busy for a couple of days, glad to see this topic inspired some brilliant posts.

    I'm going to have a read up on this asap.

    Thanks to all for the input, keep it coming.

    Props to all esp the very talented mr FAST6191.

  7. jebrown19

    jebrown19 Advanced Member

    Sep 28, 2008
    Fast -> You lost me at hello... but really that's a mean block of text... and sounds like you know your stuff. I stopped reading about 1/2 way through... so you may have already mentioned my idea, so if you did.. my bad.

    I have little wii programming experience but could you define the shape in terms of the wii-motes co-ordinates in correlation to the center of the shape ? (ie. trace the shape from its center and see if it is symmetrical and follows the general pattern of the desired shape)

    Basically my thoughts would be as follows

    defined a start point by when a button (draw) was pressed (say b).

    While the button is still pressed
    follow the movement of the pointer and store the values in a 2d vector
    when the drawing is done (ie. button is released) check for a fully enclosed shape ( has start point been overlapped)
    Then sort your co-ordinate values by Y (for a square your highest points should be at Middle of your X co-ordinate)
    now transverse down your y co-ordinates (each should be visited twice) and ensure the x values at each are equal distances from your Middle X co-ordinate (at Y Max)
    also check to see if your Widest point is at Y = Mid

    Could be a convoluted way of doing things but you could just modify the rules of allowed distances to adapt for other shapes.