Dive into a drawn world by dressing up with weird hairstyles, fancy moustaches and robot ears. Become somebody/ something else or discover a new side of yourself!

In this installation you can catch drawn (and animated) body-parts/ accessories and clothes with real post-its. You see yourself on the screen, dressed with the objects of your desire, moving in a drawn world.

More info at http://digital-puppetry.de

Tine Papendick (Berlin, Germany) studied textile design at the Academy of Fine Arts in Hamburg and worked in a puppet animation studio before studying animation at the Academy for Film and Television in Potsdam.

http://ti-pi.de

Collaborators:

Rory Solomon: programming

Cameron Browning: programming

Bubblyfish: music

Real Post-its/ adhesive Markers are tracked by a camera. (Either by a color-Key or with the help of IR-light). With these Post-its, elements of the screen can be caught. Once attached to a post-it/ Marker, these elements can be moved in front of the screen freely. This allows people in front of the screen, to dress up as they want.

The environment/ background/ surrounding of the scenery (on the screen), supports the way they people themselves. The whole experience is rounded up by a corresponding soundscape.

Technical Functional Areas / Plan

1. Basic Blob Tracking (done)
2. Color Blob Tracking (Wed)

• Progress:
  

We are working on finding good thresholds to track the stickies. Technical Steps in doing this:

• XML Save/Loading to remember our values
• Fast-scrub keys to quickly look through video tests
• Threshold Scrollers to manipulate thresholds quicker

1. Blob / Accessory Coupling (Thur)
2. Silhouette Tracking (Thur/Fri)
   • to enable placement w/in backgrounds, etc
3. Behavior Tweaking (Tue/Wed)
   • accessory disappearance latency
4. Accessory Animation (Fri)
5. Background Parallax Scrolling
6. Music composition/inclusion (Sat/Sun)
7. Music Responsiveness (Mon)

Concept/ Design Issues

1. what is the general, general bg setting?
2. how do the bg settings for each category look like?
3. how does the pick-up-elements-conveyor-belt-area look like?
4. what is the relation of sound and image?

Other Think-about-it's

1. what happens to USED sticky Notes? (leave them all on the floor? stick em in a guestbook? on a wall? skewer them on a nail?)
2. how do we lighten the installation?
3. Documentation (Screen-Capture Program? Writing Video in OF?)

Software Bug/Todo List

TODO: THURSDAY:

• lighting!
• get somebody to paint the walls
• order/ buy cardboard materials and markers
• draw content (conveyor belt, accessories)
• get somebody to finish technical setup/ cables etc.
• improvements of code

FRIDAY:

• build environment
• mount the box
• print poster/ mount poster
• finish the coding

SATURDAY:

• last configurations on animated content
• last configurations on code
• last configurations on environment

Wednesday, July 9th:

• rearranged the whole space, because the videoprojector was not bright enough. now everything fits!
• had a great brainstorming about creating a “real” faked surrounding for the installation: means: connecting the projection with the physical space (floor, background)
• finished the box
• codewise?

Tuesday, July 8th:

• lots of software progress:
• improved blob/object pairing
• xml file completed
• xml specification of images completed
• much physical space progress (camera! projector, screen)
• caught ourselves by just drifting off and playing with the software

Monday, July 7th:

• frustrating software day … not much progress … code handoff from Martí … rory/cameron trying to understand that and make improvements
• what else?
• testing several videobeamers, cameras and positions.
• mounting the hanging projection wall: it looks GREAT!

Sunday, July 6th:

• going to home depot, buying materials
• building the box! tine is trying to charm people who help her with the tablesaw! (great, it worked out in the end!)
• conceptually drawing. thinking about the spot

Saturday, July 5th:

• i am feeling sick so i decided to go for a long walk in the park instead of sitting in front of a screen
• figured out what i want to do with the used sticky notes and how i want to mark the area where people move and how i want do the technical setup

Friday, July 4th:

• testing cameras in combination with a projector. discovered, that the lighting has to be very very good!
• thinking about a possible setup and the installation of the installation
• playing around with the software
• animation tests of backgrounds/ foregrounds completed
• shooting of more test videos in the setup
• gave the code to marti and maria, they will help with w/ the code on the weekend! jippee!

Thursday, July 3rd:

• testing several cameras and camera positions, talking about technical setup w/ gustavo
• friedrich figured out how the code works. made it possible to attach one object to one tracked blob.
• playing around with the software
• animation tests of backgrounds/ foregrounds

two objects… and first test:

Wednesday, July 2nd:

• advanced color blob-tracking
• thinking about visual concept. categories. backgrounds. conveyor belt.
• getting OF to run on tines machine/ workshop of object-orientated programming.

* tine's timetable:

I Want To:

• decide for how backgrounds shift (thu,3rd)
• decide for how objects (accessories) fall/ disappear (fri,4th) when a sticky note is hidden from the camera for longer than 30 frames, objects will fall, until they disappear from the screen. they will then reappear on the conveyor belt. the same thing happens when the sticky notes are being moved too fast.
• decide for how accessories are brought onto the screen (conveyor belt movement) layout issue (fri,4th) the elements will move as on a conveyor belt, there will be just a slight difference in the background color compared to the rest of the screen. no “real” conveyor belt
• decide for the clipping of the real-time video on the screen (without feet, from hip onwards?) (fri,4th) people see themselves completely on the screen!

• decide for what happens with the used post-its (guest book/ wall?) (sat,5th)
• build the dispenser/ disposer box for post-its (sun,6th)

• decide for a technical setup (mon,7th)
• how does it feel to pick/ select accessories? what can be designed interactionwise? (mon,7th)
• have an idea about rotation, scale, animation (falling/ moving) IN CODE, communicate this to the team (mon,7th)

• draw new accessories (still and animated)(thu,10th)
• make a poster for the spot (thu-fri, 10th-11th)
• rder materials for physical space (thu,10th)
• build the physical space (fri,11th)

• bring all of my files into the right format (thu,10th)
• provide eyebeam with a good documentation of the project development (sat,12th)

Tuesday, July 1st:

• test color videos recorded with sticky notes (one with multiple colors, one with pink only, one with orange only)
• mock-up flash file done
• talking about musical approach
• getting first blob-tracking to run
• staring on code lines

Monday, June 30th:

• first tests with infrared camera/ IR-light and reflective materials.
• the reflective works -as seen with the human eye- fairly reliable, also when changing angles. blob-tracking tests shall deliver a clearer assumption.
• reflective tape is relatively expensive, maybe we can work out a solution less cost-intensive
• tracking strongly colored post-its is to be tested tomorrow

• the character of the elements that people can choose shall be divided in three categories.
• the surrounding “world” shall consist of three layer: background, middle-ground, foreground. each category has its own layers.
• each category has its corresponding soundlayers.
• depending on the elements that people decide to wear, the surrounding will change as well. for example: wearing only elements of category 1, will make background, middle-ground, foreground and the music appear with the same characteristics. wearing half category 1 and half category 2 will produce an equal mixture of both categories in sound and surrounding…
• proposed categories are:

1. 01. Hip Hop: cool, money-orientated, stylish, clean, hip, cliché, bouncing, proud, family-gang-network-oriented
2. 02. Death Metal: heavy, dirty, tattooed, drunk, loud, thunder & lightning, head-banging, dark
3. 03. Hippie: peaceful, romantic, playful, colorful, hypertolerant, political, philosophical, stoned

• each category has its corresponding soundlayers.
• elements are passing by on a conveyor-band-kind-of-thing, which is located on the top of the screen/ above peoples heads (the belt is causing a constant background-sound)
• elements can be takan from the belt by monitoring a post-it/ Marker in front of them.
• the vbelt is looping constantly.
• if the person blocks a post-it longer than a certain time from the camera-view, the element that was attached to the post-it, drops out of the picture.
• the “lost” elements appear in the line on the conveyor belt again.