Doublepoint integrates gesture recognition into consumer gadgets for better AR

Doublepoint has raised $3.3 million to integrate its gesture recognition technology into consumer electronics and other hardware.

The idea behind the technology from the Helsinki, Finland-based company is to simplify the use of hardware devices for things like augmented reality and other purposes without any learning curve issues, said CEO Ohto Pentikinen in an interview with GamesBeat.

The technology can detect if you pinch your fingers simply by using the sensors already present in Android smartwatches. The company said it can detect such microgestures very accurately and do so much cheaper than cameras, which are used in devices such as virtual reality headsets. (In fact, there are 10 cameras to detect motion in the Meta Quest Pro VR headset).

People can create devices that are significantly cheaper, Pentikinen said.

Doublepoint will officially showcase its technology at Augmented World Expo USA 2023, taking place May 31-June 2 in Santa Clara, CA. The company plans to license the technology to others.

After three years of stealth, Doublepoint has patents pending for its software platform, which works with a range of Google Android-based hardware devices such as AR headsets.

Funding came from a range of angel investors and established venture capitalists from across Finland and Europe. Doublepoint’s investors include FOV Ventures, Superangel VC, Superhero Capital, Sune Alstrup, Timo Rein, Business Finland and others. Doublepoint currently has a total of 11 employees located in Finland, Sweden, Germany and the UK.

How it works

Doublepoint currently offers a highly advanced machine learning algorithm that accurately detects touch microgestures, surface interactions and more, CTO Jamin Hu said in an interview with GamesBeat.

When I tap my fingers together, index finger and thumb together, it registers a tap, Hu said. But the key point here is that it only registers when I actually touch my fingers together, not when I try to trick it without touching it. It detects the contact moments of different fingers on various different objects. And fidelity depends on whether we have custom hardware built for it.

The technology supports the simplest and most universal gesture creators could think of. They designed it to be effortless to work in any hand orientation. This means that you don’t need to memorize a lot of gestures and you don’t need to pay attention to them.

This is all accomplished using generic smartwatch sensors and minimal computation cycles from an integrated central processing unit (CPU). The platform uses an IMU (inertial measurement unit) to detect movement. These sensors are quite cheap.

So for the smartwatch, we just need the inertial measurement unit, which means any smartwatch will do because all of them have initial measurement units, Hu said.

In addition, it uses bio-acoustic sensors. Overall, it can detect the topography of your wrists and how it changes as you move.

So, for example, when I press down on a table, you’ll see the tendons at the bottom of the wrist strain, Hu said.

The position and movement of hands in devices can be measured in a variety of ways, including wristbands, rings, radar, LIDAR, and cameras. Initially focused on out-of-the-box gesture-enabled smartwatches as well as XR devices, Doublepoint aims to solve the toughest input issues in the easiest way, Hu said.

This is accomplished by using an extremely sensitive mechanical sensor in a smart wristband to detect movement, position, and micro-vibrations.

They found it difficult to design an algorithm to only detect touch and nothing else. It shouldn’t accidentally trigger when you touch a table, grab things around you, and just go about your daily life. It should only react when you specifically pinch your fingers.

With Doublepoint, no expensive optical cameras or LIDAR sensors are needed. The Doublepoints wristband-based technology instead focuses on vibrations that only occur when you touch something, a much simpler, much cheaper and much more robust offering than other optical solutions, Hu said.

In addition to algorithmic programming solutions, the company also plans to provide custom hardware designs dedicated and optimized for touch sensing. Doublepoint offers a free Touch SDK, which turns existing smartwatches into real input devices and provides it for download on the company’s website.

Initially focused on smartwatches powered by Google Android Wear OS and XR devices, Doublepoint uses physical muscle monitoring to function and requires no computer vision sensors. Doublepoint started with the Android-based AR input issue, as there is currently no discrete and/or touch input method in production.

Partners can license the company’s algorithm and include it in their smartwatches. Once integrated, the algorithm will run continuously within the watch’s operating system and transmit the required gesture events via Bluetooth to the appropriate headsets.

Origins

Pentikinen previously founded a real estate technology company and he has known Hu since middle school. Hu is a classically trained pianist and biomedical engineer. The business started with his invention of how to type messages in the air with a wristband.

Hu began studying this technology as he tried to figure out how to read his own subconscious finger twitches. Then it morphed into the notion of measuring humans when they touched real physical objects.

We originally founded Doublepoint under the stealth name Port 6 to keep our competitors guessing and under the sole focus of gesture-enabled AR devices, Pentikinen said. Since then, we have found a ready and enthusiastic market in smartwatches as well as XR and finally revealed our production name, full funding and full mission. We are extremely grateful to our VCs for the most recent influx of capital and their supreme confidence in our team, our technology and our vision.

Other leaders include Eemil Visakorpi, developer. He previously worked as a researcher at the Quantum Computing Group at Aalto University. And there is Lauri Tuominen, ML Engineer. He previously worked at Nokia Bell Labs, specializing in machine learning.

Advisors include Sune Alstrup. Co-founder of Eye Tribe, an eye tracking company acquired by Meta. Another is Miklu Silvanto, design director at Bang & Olufsen. And Markku Mkelinen, formerly Senior Director of Engineering Partnerships at Meta.

The road ahead

Doublepoint is most often compared to wearable devices based in Israel and Coolsotech based in Taiwan. When asked if Apple’s rumored mixed reality headset could use such technology, Doublepoint replied that nothing was confirmed yet, but based on the most reliable public sources to date, the Gesture recognition seems to be of great interest to Apple for its next headset.

Doublepoint isn’t talking about customers yet, as it’s only just been released. Some rival companies make their own custom hardware, and Apple itself makes an Assistive Touch, which is recognition technology for smartwatches.

We are currently the only one providing a software solution for this gesture recognition and will license everyone, Pentikinen said.

For games, apps could get quite interesting. You can make homebrew controllers that you can 3D print, with no electronics inside, where you can make a sticker with the sensors that could detect your fingers. And so maybe for $5 you could make your own game controller. It’s reminiscent of Nintendos Labo’s cardboard devices.

As for VR devices, we just need a smartwatch and don’t need to add anything more on a headset, Hu said.