teaching machines

OpenGL and Mouse Interaction on Raspberry Pi

October 4, 2012 by Chris Johnson. Filed under buster, public.

Last time I was able to get a simple triangle up on my Raspberry Pi using Python bindings for OpenGL ES. My goal this week was to get mouse interaction going. My big question was, “Can I move the triangle?”

Normally I’d use a windowing toolkit like wxWidgets or SDL to get mouse events. But I’m not even running an X server on my Raspberry Pi! I looked at the demos in pyopengles and found that they could run without X and still support mouse interaction. Having grown up on supporting libraries, this was not a world I was familiar with. I found the details in pymouse.py.

To get mouse interaction in my demo, I followed this progression:

• I tried to understand the implied protocol in pymouse.py. Three bytes were read from /dev/input/mouse0. Ran “man mouse” and searched for documentation to find explicit protocol. I failed.
• I reverse engineered the protocol to the best of my knowledge. The first byte is ??YX1MRL. L is 1 if the left button is down, R is 1 if the right button is down, and M is 1 if the middle button is down. 1 is apparently always 1. I don’t know anything about bits 6 and 7. (They don’t correspond to wheel clicks. On those, I get an event, but only 1 is set.)
• The second and third bytes are the horizontal and vertical displacements, respectively. If the Y bit is set in the first byte, the third byte should be interpreted as a negative value. If the X bit is set in the first byte, the second byte should be interpreted as a negative value. In pymouse.py, if the X or Y bits are set, the author interpreted the corresponding displacement in a positive way and then subtracted off 256. The magic of that expression tripped me up considerably, but I eventually figured out what he was doing. If those bits are set, the displacement bytes are a negative number stored in twos complement. To convert a twos complement byte to its corresponding positive decimal value, one 1) flips all bits and 2) adds one. Said another way, ~B + 1, where B is your byte. Flipping B is the same as subtracting B from 1111 1111, since 1 take away 0 is 1 and 1 take away 1 is 0. 1111 1111 is 255 decimal. We can rewrite our previous expression to 255 – B + 1. Simplifying, we have 256 – B. We’re almost to the author’s original expression! The last step is to realize that this conversion produced the positive value of the twos complement number. We know we want the negative value, so let’s multiply by -1. -1 * (256 – B) = -256 + B = B – 256.
• I was fascinated by the fact that the mouse is a completely relative input device. Of course this is obvious, but any mapping to absolute pixel coordinates is completely done outside of this low hardware layer. I only get displacements from the bytes.
• Next I simply started up a mouse listening thread and wrapped up my OpenGL renderer with a loop that read the mouse state each iteration. Possibly I may want to rig up my own event queue and loop. The triangle’s positive became a function of the mouse position.
• I have no idea how many mice will follow this same protocol.

And it all worked!

Here’s my crude mouse listener:

#!/usr/bin/env python

import threading
import time

class MouseListener(threading.Thread):
  def __init__(self):
    threading.Thread.__init__(self)
    self.fd = open('/dev/input/mouse0', 'r')
    self.height = 1080
    self.width = 1920
    self.x = self.height / 2
    self.y = self.width / 2
    self.finished = False

  def run(self):
    while 1:
      while 1:
        buttons, dx, dy = map(ord, self.fd.read(3))

        # Bit 3 is always set. Always! Move on.
        if buttons & (1 << 3):
          break

        # Maybe not always? Maybe a read failed and bit 3 wasn't set.
        # The original author suggests we try to sync up again with a
        # read of one byte. Why only one?
        self.fd.read(1)

      if buttons & (1 << 1) and buttons & (1 << 0):
        print 'both pressed'
        self.finished = True
        break
      elif buttons & (1 << 2):
        print 'middle press'
      elif buttons & (1 << 1):
        print 'right press'
      elif buttons & (1 << 0):
        print 'left press'

      # Going left 
      if buttons & (1 << 4):
        dx -= 256

      # Going right
      if buttons & (1 << 5):
        dy -= 256

      self.x += dx
      self.y += dy

      if self.x < 0:
        self.x = 0
      elif self.x > self.width:
        self.x = self.width

      if self.y < 0:
        self.y = 0
      elif self.y > self.height:
        self.y = self.height

def startMouse():
  thread = MouseListener()
  thread.start()
  return thread

startMouse()