More info: http://projects.andrewmahon.info/digitallyFit/

Andrew Mahon (New York) studies Design and Technology at Parsons School of Design. Using computer vision techniques, Andrew's project digitallyFit will allow participants to interact with a a projected image of themselves to modify their body type in real time. Like a carnival funhouse mirror, but better, the modified images will be stored, and uploaded to a web server.

www.andrewmahon.info

Collaborators:

jeff crouse: software programming

martin nadal: software programming

maria: hardware

a checklist:

• figure out status of props
• DONE write email re. summer school thing
• write email re. website details
• write email re. wall tag
• work on poster
• work on presentation for saturday
• code:
  • FIX COLOR MAP
  • DONE add warping
  • add good prop tracking
  • figure out how to trigger camera
  • figure out idle time
  • DONE, UNTESTED sounds?
  • jeff: multithread
  • jeff: serial?

wheeeeee

i figured, 'maybe they do believe'.. and decided to make my own page:

http://projects.andrewmahon.info/digitallyFit/

more work to come!

do people really believe this: ..?

what its already tuesday? shit.

anyways, another night of frustration.

started working on a windows pc, and i cannot seem to get my capture cards working. nobody is around to help!

tomorrow:

• trip to NYU computer book store first
  • buy things! see below

• get screen setup, this involves:
  • build shelf
  • mount screen
  • house front camera
  • mount rear camera

• trip out to buy props, places?:
  • home depot?
  • dollar store for lasers
  • ????

• back to software
  • what the hell is with capture??!
  • align cameras and screen, possibly develop cropping system
  • maybe toggle one screen on off?
  • figure out damn fullscreen stuff in windows
  • zach help?!??
  • need to make some brushes? how can i do this in photoshop?
  • implement openCV transformation thing

alright, well now that we understand my obsession with lists. i will make my way, and come back tomorrow rested!

shopping list so far:

NYU computer book store: -Xbee -Breakout Board -Arduino Mini USB Board

-Small Breadboard x number of xbees

-stretch sensor? -switch? -force sensor?

starting to think about wirelessly equipping props so that they can have an on/off switch that turns the effect on or off..

new group member today: maria! hopefully she can help me flesh out this list and figure out a lot of the physiscal computing things.

possibilities:

• wiimote
  • pros: easy to dissassemble and hack into something workable
  • cons: cannot be too close to the screen
• zigbee on/off
  • pros: very reliable, probably pretty easy to setup
  • cons: possibly expensive
• radio transmitter
  • pros: less expensive then zigbee
  • cons: im not sure that i have the skill to rig this up
• laser for position
  • pros: can be nearly any distance from screen and still get position information
  • cons: laser pointer may be unsafe, or turn into a distraction
• irled for position
  • pros: does not have the same safety or distracting concern that a laser pointer might
  • cons: may not be bright enough

props

• spray bottle
  • effect: spray tan to orange
  • trigger: ??
• pump:
  • effect: enlarges part of the body
  • trigger:
• suction device:
  • effect: shrinks selected part of the body
  • trigger: ??

hmm now i have some solid interaction going.

me and freidrich just had a competition to determine wether or not the smudge brush would stay. his image is first, and doesnt use the smudge, mine is second, and uses the smudge:

what do you think?

more to come later!!!!

just implemented brushes into the displacement map.

today is to figure out how this new direction will be executed..

first think about how interaction will happen? do props make sense? how can we prevent it from being a gigantic interface for kais power goo or photoshops liquify? if we use props, how do they correlate to a position in the image, ie how can we track the props?

• some possible props:
  • mounted rulers with sliders that can resize width and height
  • some sort of implant needle or gun that enlarges part of the image
  • liposuction thing that shrinks part of the image.
  • spray tan that colors the person ORANGE
• if effects are overused, the results should become more and more negative.
• we should somehow limit the time that one can spend.
• if nobody is at the digitalFitness 300000 machine, it will start a slideshow of past results.
• it would be great to print out before/after shots as souveneirs from ones experience in the machine.
• i think that the idea of instant easy fitness through questionable means to be absurd. if we can position the project as providing instant results i think that it would be really funny. certain things that we can do to this end inlclude some sort of processing time that alludes to the work that the machine is doing in the subjects place.
• as certain effects are used, it might be funny to build backrounds with images that 'relate' to each one of the traits. maybe the images can be random to provide some sense of suprise in the finished image? maybe the image placement can be a secondary step, that takes place in a “processing phase” modifications have been made?
• if we do decide to stay with the current interaction, how can we really push it towards conveying a message? can we pinpoint where the modifications take place?

second. how will we construct a rear projection screen? [a rear projection screen can also serve a green screen] how much space do we need? can we effectively position an image on that screen?

• think about using a wiimote for to detect an irLED that is placed on the projection screen. if we want the led to show through the screen, the screen will need to be semi translucent. is it possible to make a translucent screen that will be bright enough for rear projection?

third. how much of our software can be reused? what else do we need to figure out?

• what platform do we develop on? if we use wii drivers we will need to pick a platform..
• does it make sense to move image processing functions away from open cv? to a faster library?
• color map works → needs to be images
• brushes → need to be finished
• a lot of the software translates..
• how do we save images and then print them

fourth. try to figure out a schedule for the rest of the week.

• finish color map, and brushes.
• determine direction of interaction..

the direction has officially changed. i am revaluating the original premise of the project, to look at vanity, and the modifications that people make for vanity.

rather then modifying live video, people will have a snapshot taken of themselves, and modify it using a set of tools. unsure exactly how to approach this.

mockups follow

i have decided that i need to work more on documentation. i have really been slacking at it!

anyways, over the past week i have really had my head in the software side of things. despite making good progress with software, i have neglected some of the more conceptual issues of the project.

originally, digitallyFit had to do with new york's incessant concern with vanity. early last week i strayed from that concept, and really focused on the more playful aspects of modifying ones image, and modifying another's image. to add to the playful element we decided to incorporate props, each of which will modify the image differently. modifications include displacement warping, and color warping. the choice of props lies heavily in my decision to stick with the earlier concern with vanity, or the later focus on playfullness and fun.

some potential props include: -magnifying glass that enlarges the part of image that it is over (playful) -blow dryer that smudges the image wherever it is located (vanity) -vacume cleaner that sucks the image towards itself (playful) -spray tan that changes a persons color (vanity) -collagen implant (vanity)

thinking more about vanity:

-if someone uses a tool, can we limit the modification to one part of their image. this would allow us to realy pinpoint some of the unnecessary modifications that people do in the name of vanity… -each modification should be exaggerated, to the point of being disgusting. ie. spray tan should turn someone dark orange, collagen implant should drastically enlarge ones body part.. -how can we do these things in real time, and track it to the body? -if we do go this direction,weould it make sense to modify a static image, rather then live video? this would lose the idea of being a mirror? -should we think about the issue of private/public? -before and after shots? possibly framed as some sort of souvenir from a digital fitness program? can we print these?

-what sort of things do people do to modify their bodys in vanity?

-tummy tuck:

-breast enlargement:

-spray tanning:

-what happens when things go wrong:

is there some way to emphasize things like this?

-are props really the best way to acheive this type of modification? -should i begin to consider other methods of interaction?

-can we hide the subject from their modified image? how do we deal with interaction in this case?

i think that it would be really hilarious to provide people with a printed before and after photo…

in terms of changing interaction from being focused on fiducials, it really wouldnt be so hard with the current software framework. everything really just takes a x/y coordinate and possibly rotation….

some technical notes:

each prop contains a fiducial marker that allows it to be identified and tracked with 2 coordinates and rotation. this allows us to offset the center of an effect from the fiducial marker itself. the project is built in openFrameworks, a c++ framework for creative codiing..

right now the software runs painfully slow on my own computer, so optimization needs to be done.

a screenshot:

digitallyFit is a project that explores vanity in the context of digital existence. Using computer vision techniques, digitallyFit will allow participants to seamlessly interact with a a projected image of themselves to modify their body type real time. The results will be reminiscent of a circus trick mirror, but will be created and modified in real time. Modified images will be stored, and uploaded to a web server. digitallyFit highlights social ideals, and non-ideals, allowing people to modify themselves to perfection or mediocrity. digitallyFit is built using openFrameworks, a C++ framework that enables creative coding.

digitallyFit will be realized at the Interactivos? workshop at Eyebeam in New York City between June 26 and July 12. The project will show at Eyebeam between July 12 and August 9.

digitallyFit is a project initiated by Andrew Mahon. Andrew studies Design + Technology at Parsons School of Design in New York city. He can be reached at amahon [at] gmail.com.

digitallyFit needs help. If you are confident in programming computer vision software for interactivity and augmented reality, or image processing software in C++, please get in touch and then apply here!

Interaction: The audience stands on top of a small platform (thus restricting their boundary). They see themselves in the video mirror. Using various props, they can change their look, make up, sizes and etc. Other audience can change the first audience's look as well. Playful interaction is created between the audience through the props and real time video manuipulation. Meanwhile, the message of the piece stresses how our perception of self and appearance is constantly tested and modified by common sense of beauty.

Technical to do list 1. image modification- displacement map - Martin 2. tracking individual props - markers - Jeff 3. anchoring effects to subject body

Some ideas for props/ interaction Liposuction/ Breast Enlargement Vacuum(Liposuction)/ Hair Dryer Tooth cleaning dentist tool/ Shaver Skin Tan spray/ Whitening lotions Make Up brushes/ powders

Some things to think about 1. Can the program handle multiple interaction at same time? 2. Sound effect to specific interaction?