You are viewing qdot

Sun, Oct. 14th, 2007, 05:00 pm
Lightstone: 0 qDot: 1

For all of this information in a more concise, less vulgar format: http://wiki.nonpolynomial.com/Lightstone

For the Sourceforge project with the code: http://sourceforge.net/projects/liblightstone

Drivers:
Source Code Compilation requires Windows DDK to be installed. All projects are VS2005. No documentation is provided yet, 'cause I'm lazy. Mac/Linux/Python code to be posted soon.

Lightstone Test Code for Windows
Lightstone Max/MSP External for Windows PRE-ALPHA VERSION (SOURCE CODE)
Lightstone Max/MSP External for Windows PRE-ALPHA VERSION (COMPILED)

====================

Ok kids, gather 'round, it's overly technical story time!



A few years ago, a game called "Journey to Wild Divine" was released. Instead of controlling it with a joystick, you control it with your biometrics. Specifically, Heart Rate Variance (how much your pulse speeds up/slows down over time) and Skin Conductance Level (not like, dripping sweat, but the trace amounts on the tips of your fingers). Through analysis on these two metrics, the game could teach you how to relax and meditate. Deepak Chopra was involved, obviously.

The biometrics were taken through a USB widget (the blue round thingy in the picture) that came with the game, known as the Lightstone. The game still retails for something like $150, but assuming you can find it cheaper (you can find it in the $100-120ish range on ebay, or you can do like I did and scour Craigslist until someone posts their unused copy and get it even cheaper than that), it's worth it just for having the lightstone. The Journey to Wild Divine people will even provide you with an SDK... assuming you sign an NDA with them and pay them a licensing fee if you ever make money off of what you write for their hardware.

Yeah, fuck that. I just wanna make purdy shit.

So, with a renewed source of free time thanks to the end of conference-generated busyness for a while, I decided to sit down and take a crack at cracking the Lightstone, so I could have my biometrics control lighting on my desk through Max/MSP.

First thing I did was plug the Lightstone into my Mac because it was what was in front of me at the time. There's an OS X version of Journey to Wild Divine, so I knew it worked on the mac, but I had yet to install it. When I plugged in the lightstone, it registered as an HID device (An aside for those of you not intimately familiar with the USB protocol: there's a subclass of USB called HID, or Human Interface Devices. Basically, it's a standard that makes programming drivers for joysticks and keyboards and other human interface devices super duper simple, especially since multiple programs might need simultanious access to input elements). That makes sense, two outputs, probaly transfered over like, 2 or 4 elements or something, right?


Low Speed device @ 2 (0x3D100000): .............................................   Composite device: "ST7 RS232 USB BIOFBK"
    Device Descriptor   
        Descriptor Version Number:   0x0110
        Device Class:   0   (Composite)
        Device Subclass:   0
        Device Protocol:   0
        Device MaxPacketSize:   8
        Device VendorID/ProductID:   0x0483/0x0035   (STMicroelectronics)
        Device Version Number:   0x0111
        Number of Configurations:   1
        Manufacturer String:   3 "STMicroelectronics"
        Product String:   1 "ST7 RS232 USB BIOFBK"
        Serial Number String:   0 (none)
    Configuration Descriptor   
        Length (and contents):   41
            Raw Descriptor (hex)    0000: 09 02 29 00 01 01 00 80  0A 09 04 00 00 02 03 00  
            Raw Descriptor (hex)    0010: 00 00 09 21 10 01 21 01  22 1D 00 07 05 81 03 08  
            Raw Descriptor (hex)    0020: 00 0A 07 05 02 03 08 00  0A 
        Number of Interfaces:   1
        Configuration Value:   1
        Attributes:   0x80 (bus-powered)
        MaxPower:   20 ma
        Interface #0 - HID   
            Alternate Setting   0
            Number of Endpoints   2
            Interface Class:   3   (HID)
            Interface Subclass;   0
            Interface Protocol:   0
            HID Descriptor   
                Descriptor Version Number:   0x0110
                Country Code:   33
                Descriptor Count:   1
                Descriptor 1   
                    Type:   0x22  (Report Descriptor)
                    Length (and contents):   29
                        Raw Descriptor (hex)    0000: 06 80 FF 09 00 A1 01 75  08 95 08 15 00 26 FF 00  
                        Raw Descriptor (hex)    0010: 09 01 B1 82 09 01 81 82  09 02 91 82 C0 
                    Parsed Report Descriptor:   
                          Usage Page    (65408) 
                          Usage 0 (0x0)    
                              Collection (Application)    
                                Report Size.............    (8)  
                                Report Count............    (8)  
                                Logical Minimum.........    (0)  
                                Logical Maximum.........    (255)  
                                Usage 1 (0x1)    
                                Feature.................   (Data, Variable, Absolute, No Wrap, Linear, Preferred State, No Null Position, Volatile, Bitfield) 
                                Usage 1 (0x1)    
                                Input...................   (Data, Variable, Absolute, No Wrap, Linear, Preferred State, No Null Position, Bitfield) 
                                Usage 2 (0x2)    
                                Output..................   (Data, Variable, Absolute, No Wrap, Linear, Preferred State, No Null Position, Volatile, Bitfield) 
                              End Collection     
            Endpoint 0x81 - Interrupt Input   
                Address:   0x81  (IN)
                Attributes:   0x03  (Interrupt no synchronization data endpoint)
                Max Packet Size:   8
                Polling Interval:   10 ms
            Endpoint 0x02 - Interrupt Output   
                Address:   0x02  (OUT)
                Attributes:   0x03  (Interrupt no synchronization data endpoint)
                Max Packet Size:   8
                Polling Interval:   10 ms



Except it has 28 elements.

Then I read through the information. The name of the device is "ST7 RS232 USB BIOFBK". ST 7 is the 8-bit series of microcontrollers (uCs) from ST Microelectronics. RS232 is the name of the common serial protocol. USB BIOFBK, well, that's pretty obvious. But... RS232. That's a little weird in this situation. Lots of manufacturers use RS232 to talk to their uCs because many uCs already have UARTs built into them, meaning getting serial communication to work takes <5 lines of code. The NovInt Falcon, for instance, uses the popular FTDI series of USB to Serial Conversion chips. However, this device identified as RS232 while using the Human Interface Device layer. HID is made to have elements polled as needed (i.e. looking for changes in button/wheel/whatever else values), RS232 is made to send strings of data over. This seems fishy.

Moving the lightstone over to my windows machine, it does the same thing. Identifies as an HID device, doesn't even require driver installation. I put UsbSnoop on it just to see if I could decode what was coming in. Sure enough, those aren't elements coming through. It's ASCII (You can tell this by the prevalence of byte values between 0x20 and 0x7F). Lots of times engineers will write their non-critical communications protocols in ASCII to make them simple to debug and write drivers for (Once again, NovInt did this too). Unfortunately, it's garbled for some reason, and I realize I haven't actually checked to see if anyone else has written code to do what I'm trying to do yet. Time to see what others have done before I dive in.

Next stop: the Lightstone Monitor. The LSM is an open source C# application that already basically does the work I wanted, getting the values from the Lightstone. There's even a whole forum about it. It's in version 0.83c, but that version has been removed by the author, who's also gone incommunicado. However, SourceForge has version .7, which still had exactly the code I needed. After reading through his code and some of the specs of the HID layer, I start understanding what's going on.

The Lightstone communicates in 3 messages.

  • <RAW>AABB CCDD<\RAW>
  • <SER>XXXX<\RAW>
  • <VER>XXXX<\RAW>


The only message that really matters is the RAW message; VER and SER are just identifying the hardware. The AABB and CCDD are two hex numbers. AABB in raw is the HRV, CCDD is SCL. To turn them into values similar to what Wild Divine returns in its diagonostic tests, the calculation is

  • HRV - ((AA << 8) | (BB)) * .01 (Giving you a value between 1.6-2.5)
  • SCR - ((CC << 8) | (DD)) * .001 (Giving you a value between 3-15)


Now that I've described the information we're looking for, the next question is how to get at it. This is where things get weird, and I'm happy that the LSM code was there. Since this is working over the HID layer, the serial string comes in as a "Report". Reports usually consist of individual element updates from a controller (and only elements that have changed, to reduce bandwidth), but this was using them in a bastardized way. From what I picked up from LSM, here's what it looks like:

  • Always 8 Bytes per report according to descriptor (but we get back 9 for some reason). This MUST be read as a raw report, not as individual elements
  • Byte 0 was ignored
  • Byte 1 is the size of the useful part of the packet
  • Bytes 2-8 are the actual data
  • Messages end on 0x13 (ASCII newline), receiving that character is how we know when we've formed a complete message


Here's what the code looks like (I converted the C# code from LSM to C so I could use it in my Max/MSP external later. Note that this is actually c++ 'cause it's coming from a test program and I wanted to make things easier):

std::string rawAscii = "";
while(1)
{
 if (ReadHandle != INVALID_HANDLE_VALUE)
 {
	Result = ReadFile 
	(ReadHandle, 
	InputReport, 
	Capabilities.InputReportByteLength, 
	&NumberOfBytesRead,
	NULL); 
	if(NumberOfBytesRead > 0)
	{
	 for(int ii = 2; ii < InputReport[1]+2; ++ii)
	 {
		if (InputReport[ii] != 10 && InputReport[ii] != 13)
			rawAscii += char(InputReport[ii]);
								
		// check for newline in output from lightstone
		if ( InputReport[ii] == 13 ) 
		{
			printf("%d %d : %s\n", NumberOfBytesRead, InputReport[1], rawAscii.c_str());
			rawAscii = "";	
		}
	 }
   }
 }
}


So, it's gross, but it works. At least, on windows, through the Win32 HID layer. I usually like writing my non-critical controller drivers in Python with libusb/libhid, and Max/MSP externals using pyext, making them pretty much instantly portable to any system with fairly little effort. So, I put the lightstone back on my mac, fire up python, and proceed to lose many hours of my life to getting fuckall done.

See, OS X has this thing called the HID Manager built into the kernel. Whenever it senses an HID device, it automatically claims it and lines up the HID information in a nice, queued way that's easy to read through the Carbon/Cocoa APIs. It provides a queue for each element, then you just poll those queues as needed.

I was not aware of this.

I'm sure some of you can already see where this is going wrong.

I wrote a little python program using PyHID to queue all 28 elements and read data off them as needed. And I got back complete and total gibberish. I played with every level of the program for a few hours, from the python to PyHID to the HID code to writing my own Carbon based test program. Nothing worked. Further more, I had no idea how the hell the Wild Divine people were getting this to work either.

Hit google, and after quite a bit of searching and reading up on the internals of OS X's HID layer and USB management, emailed a coworker who's AppleDevList message I'd found. His experience was that it would require a kernel extension to get anything doing anything hacky over the HID layer to work right, as you'd have to actually block OS X from picking up the device for HID processing.

He was right. Later, I found a forum post from 2004 that explained all my problems.

Just in case the site ever dies, I'm pasting the messages into this post. Both are from a poster named NikLinna. First message:





This message is for Wild Divine engineering. If anyone there wants to get hold of me directly with any questions, I'll be glad to help.

I noticed that Journey to Wild Divine installs two kernel extensions on Mac OS X, STMicroTestDriver.kext and TestDriver.kext. Both are invalid kexts, meaning they have problems that prevent them from loading--specifically, both claim to have a binary file that should be loaded into the kernel, but that binary file does not exist. Part of their configuration information (the personality) makes it into the kernel anyway, though, so that the game program can access the hardware. You can verify that these kernel extensions are invalid using the Mac OS X kextload tool with the -nt options to test a kernel extension without actually trying to load it.

Whoever is in charge of these kernel extensions needs to edit their project settings and delete the CFBundleExecutable key, since there is no kernel-resident executable for these drivers and there apparently isn't supposed to be. All that's needed is the configuration/personality information to be registered in the kernel so the game program can locate the USB node representing the sensors. (Although it seems strange to me that even this is necessary; I'm still exploring the overall situation with these kernel extensions.)

I might also point out that "TestDriver" is a really bad name for a kernel extension. It's completely generic, so it could get clobbered by some other kernel extension that gets installed, or by someone such as myself who does occasional work developing kernel extensions. I haven't yet tried moving this kernel extension out of place to see if it's really necessary for the game; I moved them both out and the game couldn't find the sensors, but either or both kernel extensions might be needed.

Why is this important? Because kernel extensions are the one area where software can destabilize the whole of Mac OS X, and a kernel extension should have no problems, even seemingly minor ones, in a production software release.

Nik





Since Wild Divine hadn't installed any drivers on windows, I figured that was the reason they'd gone with the HID solution. No driver writing mess, and no weird OS installation mess. If you remember back to earlier in this story, I'd also said I'd never actually installed the Wild Divine software on mac just for this reason (not to mention I've massively fucked the libraries on my mac (thank you very much darwinports) and need to reformat the whole thing anyways)...

Except, according to this, you have to screw with the kernel on OS X... Then the last message in the thread.




For Wild Divine engineering: I removed the TestDriver.kext file and Wild Divine starts up and plays properly. It doesn't seem to be necessary at all, and really shouldn't be installed by the game. The other extension is necessary--the game can't find the Lightstone without it--but its Info.plist file needs the CFBundleExecutable property removed.

For techie customers reading this board....

After lunch today I spoke with the guy at Apple in charge of the HID/USB subsystem of Mac OS X, and he said the other extension is masking the HID behavior of the Lightstone so that it has to be accessed as a raw USB device, which only one program can read from at a time. My guess is they did this to guarantee exclusive access to the device so that no other software can read data off it at the same time (which would be possible if the default HID behavior were allowed to occur). Removing this extension prevents the game from being able to access the Lightstone, so either they did this quite deliberately to lock out other programs, or they licensed the technology from somebody who already had working USB code and didn't want to bother with the newer and more flexible HID method. There are risks to changing something that already works, after all.

This means that the Lightstone monitor I'm working on won't be able to display info while you're playing the game, unfortunately. I might take on the task of writing a kernel extension of my own to work around this obstacle, but for now I just want to get the application done so other people can enjoy it. I'm very close to having something ready; just give me another day or two.

Nik





MY GOD. YOU HAVE TO JAM THE FUCKING KERNEL TO GET THIS TO WORK. (Update: This is apparently a somewhat common procedure)

Thing is, I'm not even sure to hold at fault for this. The Wild Divine people probably picked up some cheap-ass engineer to design their hardware, because this method of conveying serial over HID is distributed from the ST Microelectronics site.

Yes, this is the method they RECOMMEND for serial communication on their chips. The HID layer is implemented in the FIRMWARE (The PHY layer of the USB stuff is on the chip, I guess). Why this is simpler than just implementing serial on the chip in firmware is beyond me. I guess maybe they're only implementing the RS232 transfer protocol (i.e. data/stop bits, etc), but... It just makes no sense. If you google for "ST7 USB BRIDGE", you'll find people having similar issues with UPSes that've used this same method.

So, until I can pull the kext on another mac machine (next couple of days), I'm stuck with windows for the lightstone. The plan after that is to implement the lightstone driver in Python using LibHID.

Lightstone Controlled Lighting System


However, I did get all of this working as a blocking Max/MSP external, though! I'll probably switch it to flext this evening (I'm trying to port all of my finished externals from pyext to flext), but the lightstone now plugs into Max/MSP, and I used it last night to control the red (HRV) and blue (SCL) components of a light on my desk. That's where this image came from.

IMG_0082


Even with all of the pain of one of the stupidest hardware interfaces I've ever seen (which I find ever so ironic since it's from a game specifically about relaxing), this was still a pretty awesome learning experience. I now know WAY more about the USB protocol and the HID layer, and I have a neat little biometrics platform to start playing with, with my own handwritten drivers.