Gesture-Based Human Verification

Prove you're human with a wave , not another CAPTCHA.

A gesture-based way to prove a real person is there , swapping the CAPTCHA-and-2FA gauntlet for one quick hand sign, read by a computer-vision model in the browser. The camera feed never leaves your device.

Role
UX Research, Interaction Design & CV model
Context
NYU · UX & AI final project
Scope
Research → working prototype
Built with
ML5.js · 21 landmarks
Password11:47 PM
entered
Type your password
•••••••••••
2FA+45s
expired
Text me a code
phone's in the other room…
CAPTCHAtry again
Select all buses
…is that a bicycle?
GestureCAPTCHA0.4s
✌️ Peace sign
Verified human.
👆
gesturecaptcha · overview
UI overview goes here
Drop your product walkthrough / demo video in this space.
Scenario
Gesture Authentication · Scenario

You just want to prove you're human.

It should take a second. Here is what today's login actually asks of you before it lets you in.

01
done

Enter your password

The one you reused, then reset, then forgot again.

02
+30s
4·8·1

Wait for a text code

A six-digit number, valid for exactly as long as it takes to arrive.

03
detour

The phone is in the other room

So is the charger. So is your patience.

06
stuck

Still locked out

Now weighing whether the bank balance was even worth it.

05
bicycle?
?

Now solve a CAPTCHA

Squint. Click every square with a traffic light. Fail. Retry.

04
expired

The code expired

The 30 seconds ended while you were in the other room. Start over.

The frustration
You weren't hacked or careless. You just wanted in.
Every step above verifies a device or a memory — never the person.
Problem

We prove we're human dozens of times a day. It's exhausting.

Work accounts. University portals. Recipe sites. Every login throws up the same gates: passwords, codes, CAPTCHAs. Each one is a small tax on your attention. Stack them across a day and security stops feeling like safety , it starts feeling like a chore.

Security is supposed to feel safe. Instead it feels like a chore.
The gap

Every check is built to stop bots , not to work for people.

Each fix defends against the machine by handing more work to the human:

2FA
One fragile object
Lose your phone and your whole life locks , bank, email, even the laundry app.
CAPTCHA
Harder for people
They barely slow bots. AI now solves them faster than we do.
"Remember me"
It never does
So you log in again. And again. And again.
Research

I dug into why these checks fail , especially the CAPTCHA.

I didn't start with gestures. I started by mapping the CAPTCHA problem space and listening to people describe logging in , in interviews, in observation during real logins, and across Reddit threads where the frustration is raw.

01 [N] user interviews
02 Observation during real logins
03 Reddit sentiment analysis
Fill in: your exact interview / participant counts (you said you have them). Replace “[N]” above and in Testing below.
What a CAPTCHA actually isproblem map
A Completely Automated Public Turing test to tell Computers and Humans Apart , it leans entirely on sensory perception (seeing distorted text, spotting buses). That's the flaw: the audio alternative is so degraded that even people without hearing loss struggle with it (Lazar et al., 2007), and few sites ship a truly accessible option at all (Gadepally et al., 2018). A test meant to include humans quietly excludes many of them.
What people told me
“Those text CAPTCHAs with distorted characters are hit or miss. Bots probably solve these better than I can at this point.”, Interview participant
“The ‘select all traffic lights’ ones… I end up clicking extra squares just in case. I feel like I'm in CAPTCHA jail, doing it over and over.”, Interview participant
What I saw them do
  • Blurry, low-quality images caused hesitation and second-guessing.
  • Every failed attempt re-triggered the CAPTCHA , repeats drove the frustration, not the first try.
  • Tiny targets were painful on phones; people pinch-zoomed just to read them.
  • When a check felt unnecessary or invasive, people distrusted , or abandoned , the site.
Most surprising finding: the audio CAPTCHAs available today are so distorted they “sound like ghost voices” , the supposed accessible fallback is barely usable.
Affinity mapping

Everything clustered into five themes.

🔁 Over-friction logins feel like a chore
📱 Device fragility lose the phone, lose access
🧩 CAPTCHA frustration outdated & often fails first try
🔐 Security paradox want safety, distrust the methods
⚠️ Reliability gaps delays, fails, broken “remember me”
Insight

People want strong protection. They just hate friction that feels pointless.

Under every theme sat the same conflict , and it's the conflict behind every auth product:

The conflictinsight
Convenience vs. security.
Nobody should have to pick.
So I reframed the brief as one question
How do we prove a real human is present , without making that human do the work?
Ideation

I explored three ways to prove a human is there.

I sketched three ways to prove a human is present, then weighed each against friction, privacy, and how convincingly it separates a person from a bot.

✓ chosenGestureA quick hand sign read by a computer-vision model. Human, playful, on-device , and a bot can't perform a live gesture convincingly.
Haptic rhythmA vibration-pattern “signature.” Needed special hardware and was hard to remember.
AI adaptiveSilent location / behavioral signals. Powerful, but it happens to you , no consent, no transparency.
Fitts List thinking: I mapped human vs. machine strengths (de Winter & Dodou, 2011; Google's People + AI Guidebook). Let the machine do the repetitive verification it's good at; keep the human in charge of the one thing they should own , consent.
Solution

GestureCAPTCHA: wave to prove you're human.

The model21 points
A CV model I trained reads 21 tracked points on your hand , live, in the browser.

Instead of squinting at a distorted grid, you make one quick hand gesture , a peace sign, a thumbs up , in front of your camera. The model checks it's a real, live hand and waves you through.

It replaces the CAPTCHA, and can stand in as a friendlier second factor. The camera feed is processed on-device and never saved.

✌️ Peace sign
👍 Thumbs up
👋 Wave
Try the live model →
Step 1splash
Start gesture check
You're always in control.
Start gesture check
Step 2🔒
Checked on device · never saved.
Step 3gesture
Shown as an icon , never guess.
Step 4done
Verified human.
No code. No grid.
👋 A wave, instead of a CAPTCHA.
Design decisions

Each design choice came from a user mental model.

I mapped the assumptions people bring to a camera-based check, then designed against each one.

Mental model 01clarity
If I don't know a gesture's name, I can't perform it.
Show gestures as icons and demos, and let people pick a gesture they already recognize.
Mental model 02privacy
The camera could be recording me.
Say the quiet part out loud: an on-device disclaimer that the image is verified locally and never stored.
Mental model 03consent
I want to start it , not be surprised by it.
No auto-detection. A clear, opt-in “Start gesture check” button hands control to the person.
Mental model 04trust
A pop-up in a new tab looks like spam.
Keep the whole check in the same tab, embedded in the flow the user already trusts.
Mental model 05readiness
I need to know what's about to happen.
A short splash screen previews the gesture step, so the camera never turns on cold.
Testing

I put the gesture check in front of real people.

The mental models above didn't come from a whiteboard , they came from watching people meet a camera-based check for the first time.

Tested withsessions
10
Participants
Watched them attempt the gesture flow cold, with no coaching.
Resultfirst-try
99%
Passed on the first gesture
Your first-try success rate , the number to beat the CAPTCHA with.
Iterationchanged
What broke → what I fixed
e.g. “people didn't notice the start button, so I added a splash screen.”
Fill in: your real testing numbers and one concrete “this broke → I changed it” example. You said you have results , this section is where they turn the study from a concept into evidence, so it's worth being specific.
Impact

It's a working concept , here's what's real, and the bar it's aiming at.

What's real today✓ working model
This isn't a mockup. The gesture check runs on an actual computer-vision model I trained with ML5.js, reading a live hand from 21 tracked points, right in the browser. A working thing , not a pretty picture of one.

The bar to beat (today's CAPTCHA, for context):

BackgroundBaymard
0%
Fail a CAPTCHA first try
Industry benchmark , not my result. It's the failure rate a gesture check has to beat.
BackgroundForrester
0%
Abandon a site over one
Industry benchmark , the business cost of getting verification wrong.
Mineto add
99%
My first-try success
Pull this from Testing once filled in , this is the number that's actually yours.
Limitations

What this does , and pointedly doesn't , solve.

Scopehumanness
Human, not identity
A gesture proves a live person is present , not who they are. So it's a CAPTCHA / second-factor, not a password replacement. Naming it “GestureCAPTCHA” keeps that honest.
Securityspoofing
A video could fake it
A replayed clip of a hand could fool a naïve check. A production version needs liveness detection , the obvious next research question.
Accessibilityfallback
Gestures aren't for everyone
They exclude some motor abilities , the same trap CAPTCHAs fall into. Every flow keeps a “Try another method” path, so the check is never the only door.
Reflection

Trust isn't a feature you add at the end.

Training the model to recognize a hand was the easy half. The real work was getting someone to feel safe letting a camera watch them for a second , and making sure the people a CAPTCHA already fails aren't failed again. That flipped how I design: the feeling comes first, the feature serves it.

The best security might not feel like security at all.
It might just feel like waving hello.