Gesture Cam + Constentful Interface


My experimental camera makes bodily gestures necessary in capturing a discernable image. It is meant to explore consent, as they must "pinch" in order to make themselves visible.

Each finger is assigned a different color, and those colors are randomly generated per session. When a certain finger is pinched, that color is then used. Both left and right hands can pinch at the same time to use two different colors at the same time. Also, the size of the colored pixels change depending on where on the canvas the pinch commences. A pinch in the top-left corner will have the smallest size in comparison to a pinch in the bottom-right.

The meaning that emerges from the experimental camera involves gestures. Video calls are often only meant to capture the shoulders up. In today's webcam-based communications, gestures have been replaced by emojis and typed comments. This is an exploration in ways our hands can have a purpose again in digital communication. This scenario specifically uses gestures in consent, or "allowing" the camera to capture you.

My consentful interface better informs the user through an initial disclaimer screen, provides users the choice to turn on their webcam and the gesture tracking, and users are able to retract their consent to webcam access + gesture tracking.

Example Video & Images

Design Process

The following were my three design sketches:

The first idea was to have the webcam recognize a hand gesture and then present a textbook definition of that gesture. For example, the peace sign would be present a wordy excerpt from wikipedia, "The V sign is a hand gesture in which the index and middle fingers are raised and parted to make a V shape while the other fingers are clenched... in the 1960s, the "V sign" with palm outward was widely adopted by the counterculture as a symbol of peace..." etc etc. The description would be purposely verbose.

The second idea was similar to the first in it would recognize hand gestures. However, in this example, each hand gesture would correlate to a different filter.

The third option had each hand dictate the colors that are used on screen. If one hand is signalling a blue fill and the other hand is signalling a green fill, they both would be present through the loaded pixels.

Design Process for Consentful Interface

The following is the pre-implementation flow chart sketch for my consentful interface:

I "sketched" out the interface by writing out the information needed on the initial screen. Here is that sketch:


I follow the F.R.I.E.S. model the following ways:

Freely Given - The webcam is not immediately accessed and the gesture tracking must be manually turned on. Consent is not forcefully given, as they are both options that must be manually turned on within the interface.

Reversible - Consent can be retracted within the running program even if it once had been given.

Informed - The disclaimer page informs the viewer long before beginning the session on important details. There is also a tutorial video linked.

Enthusiastic - Consent is not forcefully given. Users can see what Gesture Cam can do without giving webcam access by viewing the linked video.

Specific - The program specifically asks at each step to turn on certain functionalities (webcam access consent is seperate from gesture tracking consent). Consenting to webcam access does not necessarily mean the user would consent to gesture tracking. The program is specific and does not assume what the user is okay with.

The consentful Gesture Cam recontextualizes user interaction and experience through the interface and functionality. Not only do you have to consent to webcam access for your image to be visible, but you also have to actively make a "pinching" motion. The program tries to inform the user as much as possible before they agree to anything. Furthermore, consent can be retracted, unlike many other programs.


The hand tracking functionality is the 2D Hand Model from handsfree.js.

The assignment can be found here from the Critical Computation course website.