Role: Senior Developer
Client: Ohio State University Wexner Medical Center
Team: Potion Design
Technology: Cinder, C++, OpenGL, GLSL, Node.js, Python, SQLite, WebSockets, LED, NFC, DMX Lighting, Bash Scripts, LaunchDaemons, Multi-Touch, Mesh Networks, Multi-Platform, BeagleBone Black, Embedded Linux

Overview

Forest Friends is a multi-location interactive environment that provides positive distraction for pediatric oncology patients.

Goal

Create an interactive experience that provides positive distraction for patients through various phases of their treatment.

Challenges

• Positively distract pediatric oncology patients while providing continuity through their treatment journey
• Enable hospital staff to easily maintain the installation without additional burden
• Create an experience that compliments each treatment phase

Role and Team

My responsibilities included developing the core interaction and animation systems for 3, 2D animal companions (rabbit, frog, duck) via animation and finite state machines. Each animal exhibited up to 13 unique behaviors in different situations and locations, as well as had some level of autonomy. I also built a Node.js service to create a mesh network between installations in multiple locations.

The Potion Design team consisted of 4 developers, 1 designer and 1 producer in close collaboration with a number of stakeholders and vendors.

Final Product

The experience is spread across the oncology floor and consist of 4 main areas/interactives:

Waiting Room

Patients begin their journey in the waiting room. They choose a virtual Forest Friend (rabbit, frog, duck) in the form of a bracelet with an embedded NFC chip. They can then scan their bracelet in each of the spaces and interact with their chosen companion. The experience consists of a 6-screen multitouch wall where patients can explore a dynamic, evolving, forest environment. They can play with their forest friend, interact with other patients’ forest friends and discover hidden interactions in the forest environment. Focused sound domes provide an ambient soundscape with soothing audio and interactive feedback for patients.

Exam Rooms

Each of the 3 exam rooms contain a multitouch kiosk in vertical orientation. The patients can use their NFC bracelet to bring up their forest friend and play a game of hide-and-seek. The installation also consists of soothing ambient LED lighting that complements the background on the screen. Once the examination commences, the interactive can be ‘muted’ which changes it to a calm environment and a soothing ambient light.

Induction

This room is where patients are sedated prior to their treatment. The interactive in this room consists of a projection in a dark room that plays an animation of the patient’s forest friend settling down for a nap. This provides a soothing and calming experience.

The Bell

In keeping with tradition, patients ring a bell to celebrate the end of their treatment. The bell is augmented with an ‘invisible’ piezo sensor and ARM-based microcontroller. When patients ring the bell, it sends UDP messages to all interactives on the network that set off a multitude of celebratory animations and sounds on each interactive.

Results

• The end result was well received by pediatric oncology staff at Wexner Medical Center and became integral to their activities.
• I authored, co-authored and contributed to three open source libraries over the course of the project that were adopted and used on a number of other interactive installation projects – a spritesheet block for Cinder, a basic sound management block for Cinder and a 2D scene graph block for Cinder.
• I learned a lot about animation state machines and finite state machines, I was able to utilize, apply and refine these learnings on numerous projects thereafter.
• Forest Friends won the Communication Arts, Award of Excellence.

[vimeo https://vimeo.com/161289798]

Role: Senior Developer
Client: Solid Waste Authority of Palm Beach County
Team: Potion Design
Technology: Cinder, C++, OpenGL, GLSL, JavaScript, Node.js, Express.js, HTML, Jade, CSS, SCSS, Python, WebSockets, WordPress, PHP, MySQL, Multi-Touch

Reduce, Reuse, Recycle, and Rethink is a 20+ person multitouch interactive located at the education center in the Solid Waste Authority, West Palm Beach, Florida.

The interactive consists of a 20 foot multitouch table and a large video display system.

On the touch table, tour groups can explore an interactive map to learn more about the renewable energy facility and the area surrounding it. Users are also invited to participate in 3 interactive, educational games and learn more about recycling, waste and how to be more environmentally conscious. The experience is led by an instructor and controlled via a browser-based iPad application.

My responsibilities included developing a variety of features in the interactive map, 2 educational quiz games and an iPad controller application. The Potion Design team consisted of 1 designer, 3 developers and a producer.

Role: Senior Developer
Client: Brooklyn Historical Society
Team: Potion Design
Technology: OpenGL, GLSL, C++, Objective-C, OSC, XML, Lighting, LED strips

In Pursuit of Freedom is a permanent, interactive exhibit at the Brooklyn Historical Society. It narrates the untold stories of important abolitionists via images (or lack thereof), illuminated artifacts and text to reveal relevant contemporary questions.

The experience is comprised of four sections each dedicated to the stories of 3 activists. Each section contains a suspended, 8 foot projection column, a pulley system to ‘scroll’ and suspended, light-boxed artifacts related to each story. Users can use the pulley system to ‘scroll’ through the story of each activist which also lights up the artifacts based on the narrative context.

Each pulley uses an arcade spinner controller and custom hardware. The light-boxes are fitted with LED strips that are controlled via OSC routed through a lighting controller.

Role: Senior Developer
Client: Yale University Press
Team: Potion Design
Technology: Python, PHP, MySQL, JSON, WordPress, WordPress Plugin API

Interaction of Color is an interactive adaptation of Josef Albers’ most influential book on color.

I developed a custom WordPress plugin for content editors to easily create sophisticated color studies on the backend. The system comprises WordPress, a custom WordPress plugin for creating color studies and a Python script that generates static assets and json that can be packaged with the iOS app build.

The Potion team consisted of 2 iOS developers, 1 backend developer and 1 producer.

App Store link

Role: Designer, Developer
Client: Self, Parsons MFA Design and Technology Thesis
Team: Nick Hardeman, Bruce Drummond
Technology: C++, openFrameworks, OpenGL, OpenCV, SuperCollider, OSC, Photoshop, Maya

social sqncr is an interactive audio-visual installation that aims to make people more aware of the act of creating their public identity.

To create their ‘identity’, participants use their bodies to physically interact with a pseudo-musical instrument projected in a physical space to create virtual musical creatures based on their movements. The instrument consists of eight zones that participants interact with which influence the shape, movement, and aural properties of the resulting entity.

Once complete, the system captures an image of the participant to attach to the creature which is then set free into an eco-system inhabited by similar entities. The entities react to one another, much like human beings in a social network. Participants can cause environmental disturbances by physically interacting with the projection.

The physical movements required of participants ensures a high level of investment in the process, causing a heightened awareness of their creation, and thus themselves. Watching their entity interact with other entities is a reflection of their own interaction in their social networks.

Final Thesis Paper

Role: Senior Developer
Client: Smithsonian Institution, National Museum of Natural History
Team: Potion Design
Technology: C++, OpenGL, GLSL, Objective-C, macOS, JSON

Q?rius is an interactive science education space at Smithsonian’s National Museum of Natural History.

The installation consists of a series of interactive kiosks that provide young scientists a way to discover science and nature via a series of digital tools and activities. Users can access and explore the museum’s vast collection of scientific artifacts – physically and virtually and curate their own collection of specimens in the form of a Field Book, which is accessible on the kiosks as well as online.

Beautiful People is a poster generated by analyzing and processing pixel data from Marilyn Manson’s “Beautiful People” music video. The software was written in C++/OpenFrameworks. The software analyzes histogram values for each frame in the video and generates organic elements based on these values. The size and position of each element is dictated by the average brightness of darkness of that frame in the video.

The column in the middle is the image with gaussian blur and the right column is the image with median filter – top amount = 2 and bottom amount = 4. The gaussian blur seems to be better at removing points of noise although I like the effect the median filter creates.

Source

for (int i = 0; i < width; i++){
		for (int j = 0; j < height; j++){

			int pixIndex = j * width + i;

			int i_m_1 = MAX(0, i-1);
			int i_p_1 = MIN(width-1, i+1);
			int j_m_1 = MAX(0, j-1);
			int j_p_1 = MIN(height-1, j+1);

			int ne = (j_m_1 * width + i_m_1);
			int n_ = (j_m_1 * width + i    );
			int nw = (j_m_1 * width + i_p_1);
			int _e = (j     * width + i_m_1);
			int me = (j     * width + i    );
			int _w = (j     * width + i_p_1);
			int se = (j_p_1 * width + i_m_1);
			int s_ = (j_p_1 * width + i    );
			int sw = (j_p_1 * width + i_p_1);

			//place the values in the array
			sortArray[0] = pixelsMedian1[ne];
			sortArray[1] = pixelsMedian1[n_];
			sortArray[2] = pixelsMedian1[nw];
			sortArray[3] = pixelsMedian1[_e];
			sortArray[4] = pixelsMedian1[me];
			sortArray[5] = pixelsMedian1[_w];
			sortArray[6] = pixelsMedian1[se];
			sortArray[7] = pixelsMedian1[s_];
			sortArray[8] = pixelsMedian1[sw];

			qsort(sortArray, 9, sizeof(int), compare);

			int newValue = sortArray[4];

			if (newValue < 0) newValue = 0;
			if (newValue > 255) newValue = 255;

			pixelsMedian1[pixIndex]  = newValue;
		}
	}

	texture3.loadData(pixelsMedian1, width, height, GL_LUMINANCE);

Uses frontal face tracking, the cats eyes follow the face detected. Made with OpenFrameworks.