Enabling a Quadriplegic to Race Cars

Article By : Brian Santo

CENTENNIAL, Colo. — Arrow Electronics gave its semi-autonomous motorcar (SAM) design team a little under a year to enable a quadriplegic to drive a race car. As a practical matter, they did it in less than five months.

The essence of every engineering challenge is the tension between goals and constraints — invent, improve, add (and so on) versus money, time, personnel, tools, etc. It’s not that unusual to have to account for the traits of potential users, but it’s the rare project in which user traits so completely dominate all other variables, including those having to do with hardware, software, and resource.

In fact, when it came to hardware, the SAM car design team anticipated that they’d be able to use mostly off-the-shelf technology, and they did. The challenges of the SAM car project would be defined by an end user with a profoundly limited range of motion.

There was one other uncommon constraint in the SAM car project: the stakes. An implicit element of the project goals was that the design team could not kill the driver. There’s no telling if anyone at Arrow ever said it out loud, but it is not possible that anyone involved could have failed to intuit it. Everything that the SAM car team designed, they designed explicitly with safety foremost in mind.  

It’s not as if user safety isn’t something that engineers don’t deal with all the time, whether they’re designing controls for an elevator or the life support systems in a jet or a smartphone that ideally should not blow up in users’ hands. The difference with designing any kind of system for auto racing, however, is that recklessness is baked into the endeavor — it is, to use techie terminology, a feature, not a bug — and that’s when the competitors are able-bodied.

And now it’s personal
Somewhere along the line — and this is more that went unsaid — each team member also had to have realized that those stakes were going to be personal. Any solution that they devised would require one of them to crawl into the vehicle and act as a human failsafe should something go wrong when some guy with severely limited motor control was screaming around a racetrack at over 100 mph using whatever human-machine interface (HMI) they rigged up for him.

The members of the SAM car team first convened in the summer of 2013. They consumed no small amount of time on the obligatory mechanics of becoming an actual team, evaluating what skills each person brought to the party and figuring out how to work with each other while spec’ing out the project.

They also met their driver, Sam Schmidt. Schmidt was no average driver; he was a racer who had won an IndyCar event in 1999 in Las Vegas. In the 2000 offseason, driving on a track in Orlando, Florida, he crashed his car. From that moment forward, he hadn’t driven anything other than an electric wheelchair. But he enjoyed racing so much that he’d stuck with it as a team owner, and when presented with the thoroughly unexpected opportunity to get in a car again, he was eager to take it. Arrow Electronics was determined to put him in a Chevrolet Corvette and have him run a few test laps at an upcoming Indianapolis 500.

Sam Schmidt and SAM Project team: (L to R) Ryan Hanson, Noel Marshall, Joshua Willis, Robby Unser, Will Pickard, Grace Doepker, Kevin Clukey. All but Unser are current or former Arrow engineers who worked on the project. Unser is a professional racer who served as a co-driver for Sam Schmidt (foreground). Joshua Parrish, another engineer from the Arrow team, is absent from this photo. (Source: Arrow Electronics)

Arrow’s corporate office had embarked on the project in 2012 and, by 2013, had done some extensive conceptual work with several development partners (see accompanying story). In 2013, Arrow pulled the design of the prototype SAM car in-house.

Noel Marshall was a new employee at Arrow at the time, hired in 2012 after she had earned her degree in mechanical engineering, which she’d gotten into because she had always loved cars. In 2013, she received a one-sentence corporate email — “Would you like to volunteer for a race car project?”

“Well, yeah, duh,” she had thought. “Not knowing what I’d signed up for.”

Once the membership of the SAM car team was finalized, they were told what they’d signed up for. Paraplegic. Corvette. 2014 Indy 500.

Sorry — no ‘CAN’ do
Marshall and her colleagues, a half-dozen engineers from all over North America, evaluated what had been done thus far. One of the existing ideas that the team liked was one that had been developed by Arrow partner Ball Aerospace, which had created a motion-tracking system for capturing drivers’ head movements.

The basic concept will be familiar to anyone who’s seen how the film industry uses motion-capture technology to track the movements of actors standing in for characters whose ultimate appearance will be digitally rendered, such as the alien Na’vi in “Avatar” or the chimpanzees and gorillas in the “Planet of the Apes” reboot.

In the film industry, the approach involves placing numerous dots on the actor’s face, torso, and limbs and capturing his or her movement by tracking those dots with motion-capture cameras. Arrow and Ball Aerospace adopted a modification of that approach; they used infrared (IR) cameras placed in the vehicle’s cockpit to track head motion by detecting signals bouncing off of reflective dots attached to a baseball cap that would be worn by the driver.

The communications system in automobiles is the controller area network (CAN). The SAM car team knew that the key to building a successful HMI would be the ability to integrate the head-tracking system (and whatever other subsystems that they devised) with the CAN so that they could have direct access to the car’s steering, engine, and brake systems.

GM was uninterested in the project, however, and denied Arrow open access to its CAN. The SAM team hadn’t anticipated that. Without GM’s cooperation, integrating directly with the car’s electronics — the easiest course of action — simply wasn’t going to happen. The only thing that they would be able to do with the CAN was read OBD-II data — the on-board diagnostics (OBD). It was the same level of access that a Jiffy Lube mechanic has.

The only alternative left to them was to create a drive-by-wire system.

GM’s disinterest included doing nothing to make a car more readily available. Arrow would have to get in line to place an order and wait for delivery like any other customer. Arrow bought a 2014 Corvette in December 2013 and the SAM design team got the car in January 2014. The Indianapolis 500 is held in mid-May.

As they waited for delivery, they started investigating who could make some rotary actuators for them — one for the steering wheel and another that would alternately activate either the gas or brake pedal. The HMI equipment would communicate with the actuators, not the car.

They found Electronic Mobility Controls (EMC), a company that specialized in what it branded advanced electronic vehicle interface technology (AEVIT). When the Arrow team received the Corvette in January, they immediately shipped it off to EMC in Augusta, Maine.

At Arrow’s behest, the EMC team provided:

• a steering servo
• a gas/brake servo
• a passenger gas/brake joystick
• a driver selection button — used to transfer control between driver and co-driver
• the actuator control system (placed in the trunk of the car)

EMC returned the car several weeks later, just as February was giving way to March.

When they got the car back, the team shifted into high gear (Editor’s note: I’m not sorry about that at all). They decamped to Indianapolis, where they would have access to a speedway and where Schmidt had a garage out of which they could work.

They had to figure out exactly what would be sending signals to those actuators — what kinds of interfaces? What kinds of sensors to control steering, acceleration, and braking? Their starting point was Schmidt’s wheelchair.

“It had certain controls he could use to go forward, go backward, turn around,” said Marshall. “He was using his head to tap a sensor bar. We thought, okay, maybe we could do something like that.”

HMI beginnings
The team members had spotted the IR camera concept for steering. The question was how to use it. They decided that the driver would tilt (the term used by physical therapists is “bend”) his head toward his left shoulder to go left, toward his right shoulder to go right, and center would be center.

They bought some off-the-shelf infrared cameras (from OptiTrack in Corvallis, Oregon) to mount in the Chevy’s cockpit, got a baseball cap with reflective dots, and made the necessary modifications to the software that came with the cameras. (Later, the team would discover an intermittent but alarming problem that required another critical modification to the camera software.) The 3D tracking system would be fed into the drive-by-wire steering mechanism.

The team realized that they could also use the camera system to track Schmidt’s movement as he tapped his head backward, similar to the way that he was already using his wheelchair. With the 3D-motion tracking, there would be no need to create a sensor bar to position behind his head. When driving the SAM car, it would be one tap for 10 mph, two taps for 20 mph, and so on. The algorithm for that was similar to cruise control.

For braking, the Arrow engineers embedded a force sensor in a mouthpiece that Schmidt could hold in his teeth. How much he slowed was proportional to how hard he bit.

“It wasn’t perfect,” noted Marshall. “At first, we could only go left, but we were going to be on an Indy 500 oval, so that was enough.”

The SAM car design team installed the actuator control systems and the human machine interface (HMI) control systems in the trunk of the Corvette. Paralyzed former IndyCar racer Sam Schmidt drove this car at speeds in excess of 100 mph at the Indianapolis 500 in 2014. (Source: EDN)

The team also began creating a path-guidance system based on GPS that would automatically keep the car on course. They’d be able to plot a course down the center of the track, and if Schmidt lost control or if the car began to veer too far from the center of the track, the guidance system would immediately kick in and compensate. The team likened the system to “virtual bumpers.”

They had the actuators in place. They had built the camera and force sensor HMI and the input systems and connected them, along with redundant systems for everything. They had also had the rudimentary system devised for the co-driver, who would occupy the passenger seat. The co-driver worked a simple joystick — move it forward, the car would accelerate; move it back, the car would brake — and if the backup driver had to do any steering, he or she would have to just reach over and grab the wheel.

They tested it themselves on the track, and everything worked. So they built a simulator and invited Schmidt to try it. “It was so cool to see him in the simulator and to watch him drive,” said Marshall.

The racing line
For the first couple of minutes, anyway. The team members watched Schmidt with mounting unease. “When we saw him in the simulator, we were like, ‘Oh, shit, he’s hugging the corners,’” said Marshall.

Racers drive “the racing line” — an ideal path that allows them to simultaneously maintain maximum possible speed over minimum physical distance. Driving a counterclockwise oval, the racing line runs along the right edge of the track on straightaways, and when steering into a turn, racers want to cut across lanes to get to the left edge — that’s hugging the corner. When they emerge from the turn, the line cuts back across lanes, returning to the right edge to do it again with the next curve.

The SAM team had made a fundamental miscalculation, and it didn’t have anything to do with the technology. Everyone thought that Schmidt wanted to drive again. They were wrong. He wanted to race again. That was going to complicate things a bit. The GPS system remains in every iteration of the SAM car, a vestigial remnant of caution sacrificed on the altar of racing, there to be invoked should it ever be needed for street drivers. (It eventually would be.)

On to the next step. They brought the car to the speedway, helped Schmidt into the car, and got ready to go. And that’s when the co-driver system simply stopped working. Schmidt was behind the wheel, idling. When the speedway owners finally understood exactly what was going on — Paralyzed driver? Inoperable failsafe? — they got alarmed and threatened to rescind permission to drive their track. It was pouring rain, and the SAM team was getting soaked trying to figure out how to get the damn co-driver system to work again.

“It felt like hours,” recalled Marshall. “It was probably only 10 minutes.” They figured out how to fix the problem, the track operators were mollified just enough, and Schmidt took off and did one entire lap at 30 mph.

It worked. Conceptually, for sure. And mechanically, mostly.

That said, it was very much a prototype. “We referred to it as ‘janky’,” said Marshall. The co-driver apparatus was rudimentary, and the handoffs from driver to co-driver and back weren’t smooth enough for anyone’s satisfaction. No one, least of all Schmidt, found the head-tilt steering intuitive. The same for the head-tapping for acceleration.

The team had always thought about gas and brake separately, but finally, one of the engineers had a minor epiphany. If there was only one actuator for brake and gas, why not create a single device to control both?

There were any number of possibilities, and they still had some time for experimentation. This was when the team invited a collaboration with Freescale Semiconductor (Freescale would be acquired by NXP Semiconductors the following year, in 2015). Freescale not only is renowned for its sensor technology, but it also had amassed a lot of experience with the automotive market.

One thing that the SAM car team and Freescale tried was modifying the mouthpiece concept. They created another mouthpiece in which no pressure would translate to full braking, while acceleration would increase with the force of the bite. They tried a balloon-based apparatus that would sense air pressure. They created a mouthpiece that had two flip sensors that a driver could flick with his tongue.

Schmidt wanted to use a device that paraplegic wheelchair users were already familiar with — a straw that would detect when the user was inhaling or blowing — a sip/puff tube. This is an apparatus based on a pressure sensor (not an airflow sensor, as might be assumed).

Marshall didn’t like the idea. It seemed inelegant and potentially prone to error — a gasp or even a stray breath might result in unintentional acceleration or braking.

Nonetheless, “we tried it, and it was our favorite,” she said. At first, it was sip to accelerate and puff to brake. Along the way they reversed that; it’s now sip to brake and puff to accelerate.

Brian Santo in the SAM Car simulation. Drivers of the SAM car steer by turning their heads in the direction that they want to go. The head-tracking system uses infrared cameras to look for reflective dots on a hat worn by the driver (Arrow’s simulator for the car embeds the dots in a pair of glasses). Drivers brake and accelerate by inhaling and exhaling, respectively, through an air tube equipped with a pressure sensor — the commonly used term for the apparatus is “sip/puff.” (Source: EDN)

[Editor’s note: When I got to use the simulator when preparing for this story, I found the sip/puff mechanism obvious and easy to use, though after a few minutes (racing simulators are fun), my cheek muscles began to ache. Also, for the simulator, the hat has been replaced with lens-less eyeglass frames studded with reflectors. Elton John would be envious.]

Schmidt was happy enough to ditch the bite sensor. He had previously taken a picture of himself in the car making a deliberately goofy face, with a tangle of wires trailing out of his mouth, and captioned it: “Would you go on a date with me?”

Steer it up
The steering system, however, was proving an intractable problem. The major struggle was with mapping the relatively limited range of head bend to the wider range inherent in the steering system.

Adults with a good amount of flexibility can bend their head 45 degrees in either direction for a total of 90 degrees. The average steering wheel, meanwhile, has a steering angle of 540 degrees — it can spin around 1 ½ times. The relationship between the two would be complicated by having to also account for the steering ratio, which is the amount that a steering wheel turns versus the amount that the wheels of the car actually turn. Different vehicles have different steering ratios, ranging from about 12:1 to 20:1. So if you have a car with a steering ratio of 16:1 and you give the steering wheel a 360-degree spin, the wheels of the car might turn 22.5 degrees.

As a practical matter, anyone who was going to drive the SAM car — not just Schmidt — was unlikely to be flexible enough to get the full, theoretical 90 degrees of head bend. Whatever the actual range of head movement, it was going to map poorly against any car’s steering angle; a fairly modest head bend could result in a disproportionately large response from the vehicle. And it was simply unreasonable to expect any driver rattling around a speedway at 100 mph to be utterly precise with their head bends. The team tried to compensate, endlessly readjusting the system. One of the engineers would get back in the car and try to do a figure 8 and fail again and again.

On top of that, once the team was driving out in the world, an unexpected problem with the head-tracking system arose — literally arose once a day, every day. The infrared cameras would occasionally detect rays of sunshine cutting in through the vehicle’s windows and misinterpret them as reflections from the driver’s hat. Compounding the problem, the cameras sometimes caught glints off of some piece of jewelry or some other reflective surface and misinterpret those as well.

It all amounted to plenty of impetus for the SAM car team to ditch the IR camera system and replace it with something else, and it just so happened that partner Freescale had something else to suggest: an inertial system based on one of its own inertial measurement units (IMUs).

IMUs combine gyroscopes, accelerometers, and, in some, magnetometers (this one did). The idea would be to mount an inertial measurement module on the driver’s head and interpret that data for the steering and perhaps the braking and acceleration, too.

The Arrow team and its colleagues from Freescale designed and built a module and tried it out in the SAM car.

It didn’t work — not at first. The unit was failing to find true north (probably because the cockpit was too noisy). The module needed true north for a zero reference, and lacking it, the system would drift. The practical effect was that the SAM car didn’t always go where the drivers thought they were telling it to go. Freescale diligently kept at it, however, and finally got it to work — but not before the middle of May rolled around. The IC manufacturer proved the concept but not in time for the Indy 500 demonstration.

Turn, turn, turn
The Arrow team, denied their most likely possible alternative, redoubled efforts to fine-tune the infrared camera system.

“We were at it for months,” said Marshall. Finally, the same engineer whose suggestion led to combining the braking and acceleration into a single interface device suggested tracking a different head motion — swiveling the head (technically, head “rotation”). Drivers would rotate their head right and go right; they would rotate their head left and go left. Basically, you looked at where you wanted to go. The average person can rotate their head close to 90 degrees in either direction for about 180 degrees total. Switching from bending to rotating practically doubled the range of head movement, making it much easier to map head motion to steering angle. The practical result was that the driver could get away with head movements that were far less precise.

“It was like a light bulb went on,” said Marshall. “Figure eights? It looked like we were driving with our hands and feet. We put Sam in it, and you’d never know he was paralyzed. That was it. It gave us more control, and it felt more intuitive.”

As for the problem with stray shafts of sunlight? The team blacked out the windows, began wearing matte black suits, and took other measures to eliminate spurious flashes, but those were always going to be short-term solutions. Earlier, a critical software update was mentioned, and this was it. Once the team figured out what the problem was, they were able to add filters to eliminate signals that were, essentially, noise.

“Now we can drive with the roof off,” said Marshall. “It’s perfect. Actually, that’s a weird word for an engineer to use. Let’s say it’s pretty damned good.”

The events associated with the Indianapolis 500 are spread over a week. On May 18, 2014, during one of the first events at that year’s Indy 500, Sam Schmidt drove two laps, getting up to 80 mph, driving the Arrow SAM car. He did four more laps later in the week. On the Monday following the race, he took another test drive on the track and hit 106 mph.

Two years later, he returned to the same track with an upgraded SAM car, did four laps, got up to 130 mph on several straightaways, and on one stretch, he edged his speed up over 150 mph.

A few months after that, Schmidt participated in the annual race up Pike’s Peak, a time-trial race on a course filled with esses and hairpins. It takes an average driver roughly 45 minutes to traverse the course under ordinary driving conditions. Schmidt made his run in roughly 15 minutes, which put him in the middle of the pack of competitors. He had beaten a good portion of able-bodied competitors in the race.

In 2017, he travelled out to Nellis AFB in Colorado for the Air Force’s annual Aviation Nation Air Show, and — in a race with a jet taking off — he pushed the SAM car to 190 mph.

Street legal
The latest iterations of the SAM car have a much more sophisticated drive-by-wire system for backup drivers, including a co-driver set up that includes backup steering wheel, gas, and brake pedals. The handover, accomplished with a simple button click, is smooth and immediate. For the new version (originally designed and built in 2016), the SAM car team contracted with Paravan, a company in Germany that specializes in drive-by-wire systems.

EMC, which had built the original actuator subsystem for the 2014 version, wanted to limit its participation to building only that subsystem. Paravan was willing and able to integrate the drive-by-wire subsystems that it created for the 2016 version with the sensor-based HMI subsystems that the Arrow design team had created, leading to a more smoothly operating whole. It had also redesigned the whole system with triple redundancy. From a physical layout standpoint, the 2016 version was better organized. Paravan had also straightened out what had been, in comparison, a rat’s nest of wires.

“German engineering,” said Marshall admiringly. (Now that she’s been out of college and can afford a new car, she’s bought herself a series of Volkswagon Jettas. Go figure, right? “Manual,” she specified. But of course.)

The new SAM car includes voice control for auxiliary functions (wipers, blinkers, etc.) and for the shifter. An after-market camera system has cameras trained on the vehicle’s blind spots, with a display mounted prominently on the dashboard — after all, SAM car drivers can’t turn to look at their blind spots because if they do, they turn the car.

Some of the new features, including the voice activator for the shifter and the blind-spot cameras, were adopted to enable disabled drivers to get driver’s licenses for street driving. Arrow corporate has worked with the Department of Motor Vehicles in the state of Nevada on that.

We asked Marshall if there was anything else about developing the SAM car that she wanted to add. She said, “This team is a family. What we went through — everything was hard. But it was a once-in-a-lifetime opportunity. We might never have anything else as good as this.”

“Always a Driver” by Aaron Hecquembourg features an image of IndyCar racer Sam Schmidt, the demo driver for the SAM car prototype. The artwork hangs in an Arrow facility near Denver. (Photo: Brian Santo, EDN)

Arrow Electronics is the parent company of AspenCore, which publishes EE Times, EDN, Electronic Products, and other publications.