In close collaboration with the game development team, NERD developed a heart rate detection system that uses the Joy-Con’s IR Motion Camera. Pulse detection is performed entirely on the Joy-Con’s microcontroller unit, then sent over Bluetooth to the main device with no additional bandwidth cost, thanks to a new NERD-developed protocol.

This custom protocol was also used to send compressed Joy-Con accelerometer and gyroscope data in a highly reliable way, improving the responsiveness of the player’s virtual position. The Joy-Con firmware was updated to include these changes, enabling some of the new features found in this fitness-filled quest.