Agency of Emotion
[AI with EQ]
Engaging with provisional realities cedes the human voice in favor of predictive algorithms. Will this improve our relationships, or produce an unacceptable strain on our individual agency and identity?
Agency of Emotion is a speculative software which embeds text based provisional realities into existing platforms, such as instant messaging and email. Provisional realities are formed by combining human emotion and temporal augmentation (predictive simulations). The resulting natural language generation appears plausible, because it is based upon an ever-increasing dataset of previous user interactions (and their emotional effects). Agency of Emotion interrogates: emotion based identity, roles of system transparency, and the collision of technology with emotional intelligence.
I watched a captivated crowd applaud with joy during a developer conference in 2018. Google Duplex was being demoed, and my stomach turned as I listened to an AI digital assistant talk to people and book appointments - while it passed for being human I saw it as creepy and deceptive. But why? This interest evolved into the question:
How might AI generated content influence, or shift, the relationship between agency and identity?
Fast forward to today. As agentive technologies have become increasingly pervasive, so to has insertion of human emotion in these experiences. Algorithmic empathy, and improvements in natural language processing are present on a growing number of digital assistant platforms. Imagining the future of these systems reveals a complex space in need of critical design. For this reason, my primary audience are designers whose work overlaps with combinations of: natural language processing, predictive simulations, and designing for emotion.
The two preceding video scenarios imagine a near future where speculative software operates within existing platforms. The embodiment of algorithms with emotional intelligence create what I call provisional realities. These derivative outputs are analogous to forms of skeuomorphism found in user interfaces. Provisional realities are formed by combining human emotion and temporal augmentation (or predictive simulations). The resulting natural language generation appears plausible, because it is based upon an ever-increasing dataset of previous user interactions and their emotional effects. As a consequence of engaging with provisional realities, users cede their human voice in favor of predictive algorithms.
It is safe to say, many people today are living in a fragmented timescape. Temporal augmentation is used to facilitate and automate the continuing rise of asynchronous human to human communication. Users have the ability to swipe and modify the degree of data history used to generate suggestions, which results in use of different emotions.
timescape. Noun. (plural timescapes) (science fiction) A multi-dimensional treatment of time. (physics) A function of time that is dependent on the position of the observer.
I leveraged Robert Plutchik’s “Emotion Wheel”, (pictured in the center of the following image) to categorize and root the framework of emotions. This also helped to subjectively pair them in scenario videos. Surrounding the diagram are UI cards which reveal to system users a superficial level of emotion source disclosure, alongside the primarily text based chat.
For me this inquiry began with Google Duplex, and evolved through the insertion of emotion, and concept of temporal augmentation. As opportunities for human connection are further automated, there comes increased risk of emotional detachment and homogenization of expression. Engaging with provisional realities cedes the human voice in favor of predictive algorithms. As an audience of designers, technologists, and humans, I challenge you to consider the question as you move forward in your own practice: Will this improve our relationships, or produce an unacceptable strain on our individual agency and identity?
For a deeper dive into research, past prototypes, and additional documentation...
THESIS PAPER (click for PDF)
ZINE: INSPIRATION + PROCESS (below)
ULSOCC
(Ultimate Live Streaming Obstacle Course Challenge)
ULSOCC is a new obstacle course live stream created by adding mocapped YouTube exercise video motions onto digital avatars in a simulation, pitting motion against motion to see who will come out on top.
Follow on Twitch at: https://www.twitch.tv/ulsocc/
“American Ninja Warrior”, “Wipeout”, “Ultimate Beastmaster”, and “American Gladiators” are shows designed to exhibit human physical accomplishments. Likewise, ULSOCC embraces arbitrary obstacle architectures invented for military mental and physical fitness training, and challenges participating humans to bend, crawl, and jump their way through courses.
What are design opportunities related to the future of digital identity ownership, access, and agency?
Research and making for this project are broken thematically into four sections below:
This project blends the concept of surveillance as art and entertainment alongside notions of surveillance as governmental and capitalistic data collection - power structures acting in the interest of maintaining and spreading authority. I present a version of “hyperreal motion” referencing pieces of motion from the real world, while simultaneously severing them from the original.
Playgrounds and military training grounds were an inspiration for the course development, including:
material paths: (sand, rubber, woodchips, textured) to guide, not prescribe different sections
elevations: slides, ropes, walls, steps, slopes, ladders
possible actions: crawl, spin, jump, climb, balance, swing
Several athlete body forms were tested and developed for use. In the end, all my designed athletes were derived the form I built in FUSE and named “Brutus”. The athlete uniforms linked to the particular movement specialties, related to source animation videos, or are based on a variety of common sport sponser companies.
Animations for the athletes are constructed from pirated motion capture using a variety of exercise videos from YouTube. Using RADiCAL Motion software, the MoCap data was harvested and ported to Unity where the animations were rigged onto Brutus.
Through this process I explored the right of publicity, sometimes referred to as personality rights. This refers to the right of an individual to control the commercial use of one's identity, such as name, image, likeness, or other unequivocal identifiers.
Building out the obstacle course in Unity involved a great deal of trial and error. This was in large part to balancing the purity of the animation with the game engine physics. This was particularly the case with assigning properties of motion constraint colliders.
TOOLS
-Unity
-RADiCAL Motion (AI-Powered 3D Animation)
-Adobe Fuse
-Adobe Photoshop
-Twitch
-OBS: Open Broadcast Software
CLASS
-Body Tracking
INSTRUCTORS
-Jenny Rodenhouse and Elise Co
Unscripted Interactions
OVERVIEW
Telepresence in live streaming has provided a platform for chance interactions and anonymous control, particularly through the use of chat. Societal obsession with these forms of meta entertainment range from the world of fantasy sports to live competitive online gaming. Unscripted Interactions is a demonstration and critical observation of these virtual ecosystems, resulting in both social cooperation and disharmony.
CONCEPT AND PROCESS
In my livestream project, each of the four main cars in the arena correspond with a different channel in Twitch, the live streaming video platform which boasts 15 million daily active users. These otherwise unrelated channels have their video content projected onto the arena walls. In the meantime, the chat from these channels is scanned by code in Unity, where the game is built, looking for keywords which trigger movement of the cars. Viewers of my livestream have to option to purely observe, or go to the other unknowing channels and participate in the chat, using keywords to manipulate movement of the cars in my stream. The result is a demolition derby style game fed by unknowing participating channels and anyone watching my stream who wishes to utilize these chats to control the cars.
Click images below to view stills from the final Unity prototype
REFLECTION
Throughout my ideation and prototyping there were consistent motivating interests I kept in mind while designing. I felt fortunate to see upon frequent reflection that while the outputs of these interests continued to evolve, it was in such a way that remained true to the original attraction.
Digital Ecosystems: These are micro-worlds which contain content such as eSports and other livestreaming video and chat.
Anonymous Interactions: This involves my curiosity around how might people interact with one another as strangers. It also includes remote interactions and the question - How might level of engagement and type of interaction relate to anonymity and distributed location of users?
Dynamic Closed Loop System: I was drawn to the idea of setting up a system and creating rules allowing me to run a livestream in which people might be equally interested in pure observation or getting involved to influence an outcome.
In future iterations of this project I would like to experiment with specificity around the type of keywords used to trigger the cars. For example, language specific to the type of channel initiating movement, rather than generic terms.
On the technical side, it was rewarding to get deep into Unity, the game engine software, on this project. Any next project I build in the future using Unity will benefit from this.
Below: select process documentation
MATERIALS AND TOOLS
-Unity
-C#
-Gameshow
-OBS
-Webcams
-Adobe Premiere
-Keynote
PROJECT BRIEF
In this studio, we will be considering all aspects of the our new “on camera” existence as subjects for design. The experiential elements of your online presence, your face and body, your in-and-off camera environments - these are all being rethought and remade according to a new brief that addresses not just one, or dozens, of users, but audiences of thousands or millions.
After quickly considering some historical precedent and perspective on this topic (and there is quite a lot: including traditions of theatricality in architecture, costume/masks/makeup/drag, evolutions of metaphor in UX design, set design in film and TV, live-edited sports broadcasting, early experiments in live hosting on the web, etc.) we will survey current and emerging manifestations, from live-hosted massively multiplayer online role-playing games, Twitch channels, IG Stories and Snap and FB live, and more.
At the center of this studio will be making. All students - individually or in teams - will be expected to throw themselves into some aspect of this topic, resulting in an effectively realized project for final review by outside jury, including some form of “live” manifestation to be shared worldwide.
CLASS
Live Styles of the Live and Streaming
INSTRUCTORS
Tim Durfee
Jenny Rodenhouse
Impressions of Water
(Above: selected clip from Impressions of Water. Music: Svefn-g-englar by Sigur Rós)
OVERVIEW
Impressions of Water is a short film exploring one person’s memories involving water in the Los Angeles, California area. My inspiration for this project comes from my love of natural spaces, especially the mountains. There is great irony that Los Angeles is consistently in and out of drought status while being geographically up against the Pacific Ocean. Due to this fact, water is deep in the consciousness of those living in the area. For this film, I look at water through a different lens, represented by the patina treatment of the video.
(Above: Impressions of Water - full film. Music: Svefn-g-englar by Sigur Rós)
PROCESS
To capture video for this project I travelled to over a dozen locations in the Los Angeles area by car, bike, and on foot. The video was distilled down to short clips and exported as over 16,000 images, which were made into contact sheets and printed. The contact sheets were then physically marked up and the individual images were rescanned. The remainder of the process took place in Adobe Premiere and consisted of arranging the stills into groupings based on location. For example mountains, fountains, and home were three of the categories that were used and mixed in with the underwater clips. To complete the film, the arrangement of the stills was further refined to absorb the rhythm of the music backdrop.
REFLECTION
I hiked 15 miles, biked nearly 20 miles, and drove 50 miles to capture the footage for the film and documentary. Shooting in such varied locations was time consuming but rewarding. These were all places I have been before, but was able to visit with a critical eye and make unique memories distinct from the artifact of the film. Through this project I was also able to refine my process of collecting and editing video. At the end of it all, this was a lovely excuse to look at some of my favorite places in an interesting and intimate way.
DOCUMENTARY
Below is a short documentary on the making of this film.
MATERIALS AND TOOLS
Canon EOS 5D DSLR camera and tripod
iPhone 5S
Adobe Premiere
Adobe Photoshop
Various pens, markers, scotch tape, color printer, flatbed scanner (all for patina effect)
PROJECT BRIEF
Part 1 - Film
Objective: Produce an original short film from conception to realization. The piece can be narrative or experimental non-narrative. You are free to create any kind of moving image work, about any subject matter. Criteria for a successful project are:
-Follows the rules of basic videography (exposure, etc.)
-Sensitivity to editing, proper lighting and framing/composition
-A discernible and intentional structure
Part 2 - Documentary
Objective: Create a short documentary/video journal on your practice. Use the Machine Project Vimeo documentation videos as reference.
Criteria for a successful documentary are:
-Follows the rules of basic videography (exposure, etc.)
-Sensitivity to editing, proper lighting and framing/composition
-Utilizes a DSLR (or higher quality camera)
CLASS
Dev Projects II - Media Production
INSTRUCTOR
Kate Parsons
Healthy Privilege
Healthy Privilege from David Chan on Vimeo.
OVERVIEW
Healthy Privilege is an innovative private healthcare company leveraging patent AI smart lens technology to establish a dynamic user profile by tracking behavior and habits. The collected information is analyzed in real time against a wide range of data sets, then translated into an individual health score. The health score serves as the basis for plan rates and available coverage. Additionally, the scores are shared with external service providers to unlock new opportunities ranging from job offers to new relationships.
CONCEPT DEVELOPMENT
In order to develop this project concept, I initially followed my interest in AI ethics and morals. There is currently much debate over how to handle control of this delicate subject. Depending on who you ask, control over AI might involve:
-Government policy and regulation
-Individual corporations or collective industries leading the way
-The programmer or designer of the AI should have the power to influence
I took a deeper dive and developed a speculative concept of a “Moral AI Policy”, which would be created by AI. In order to form this policy, your individual system must first train through observation of moral and ethical decisions you make in your real-world environment. The data surrounding these decisions and related interactions are captured and used in a form of reinforcement learning to form your unique policy.
Next, this policy data considers the individuals current state or situation and runs through a type of Markov Decision Process (or MDP) which provides a mathematical framework and outputs, then applies a policy decision which corresponds with what it determines to be the “best” decision for you.
Below is a diagram of what this process of Moral AI Policy training and human use might look like.
Eventually momentum for this idea slowed as it still required a context to live within. To regain traction I looked back to my previous project Knock-knock: Useless AI. Using this project and the research above as inspiration, I was able to reinvest in familiar areas of interest in addition to moral and ethics. These additional areas are digital identities, and systems (in this case pertaining to explainable AI). I will touch on each of these three areas of interest below, and later in the reflection section I’ll share key insights I came upon throughout the making and reflection process.
MORALS AND ETHICS
While exploring the Moral AI Policy, I was looking at replacing or assisting human decisions and knowledge to help manage moral and ethical scenarios. Further, this was an examination of human relationships and communication with AI. As related to Knock-knock, I was exploring the creation of systems that could be simplified as being referred to as a “set it and forget it” regarding human agency in AI. This presents the question of, “Who is in control (in power) and who stands to benefit?”. This research and inquiry came with me to the Healthy Privilege concept.
DIGITAL IDENTITIES
Healthy Privilege is built on the idea of omnipresence and data collection to feed AI. This has a direct connection to a topic I have previously explored and written about, Digital Identities and Digital Footprints. I was also engrossed in a similar mindspace last semester in the Livestyles class project where I investigated the behaviors and motivations of people who livestream their daily lives for strangers.
Not only does Healthy Privilege have a scaled up version of data collection, but in the mid stages of concept development came the idea of data leakage. That is, information gathered for one purpose finds itself shared with external services. This expanded the range and implications beyond healthcare, and into any imaginable facet of one's life.
SYSTEMS (explainable AI)
I have grown an affinity for designing concepts where one sets an task for an AI with resulting potential for unpredictability and unexplainable behavior. This was ongoing inspiration for Knock Knock, and so it felt natural to keep pushing in this direction. As with Knock Knock there may be a certain charm to dumb-smart AI. However, when the stakes are higher and there are more impactful consequences, as with Healthy Privilege it is reasonable to want to know why something happens. Unexplainable AI is wide, from concerns about reliability and safety, to wanting transparency and accountability, and attempting to determine whether there has been data bias (or rather how biased). I think at the heart of all of this is trust. As a human is can be difficult to trust something that cannot be explained in a way that makes sense to me. This gets more complex when considering machine intuition, since absolute binary decisions seem to be something we expect of machines but cannot ourselves produce. Therefore, I am interested in understanding what it is we hope to achieve by achieving explainable AI. This is the promise of the “new learning process” that Defense Advanced Research Projects Agency (DARPA) is striving to develop.
MAKING PROCESS
On the technical side, the backend for this project was similar to previous projects. Once the framework for a concept was in place, next steps for creating this project involved iterative prototyping and testing by jumping back and forth between two core areas of interaction:
1. Digital Interactions: I focused on playing out the concept with the IBM Watson (the chatbot) in order to refine conversational nuances. In order to get started with the Watson system I took an online course about building chatbots and participated in a classroom workshop. Later in the process, I created an interactive voice bot to act as the personality of Healthy Privilege.
2. Physical Interactions: I built out and tested health tracking scenarios in a real-world context in order to refine body and verbal language interaction. This helped in decision making of how to film scenarios to tell the story.
There was a good deal of experimentation in the storytelling aspects of the video output for this project. This was due to the complexity of shared space for the human and AI to have interactions, while still allowing for the message of the AI research to come through. Ultimately I am satisfied with the results, though I do think there is room for balancing refinement in terms of presenting with clarity without being blunt and overly obvious.
Initial sketches: Building the chatbot with Watson Assistant Testing the voicebot interactions Creating the Healthy Privilege video
REFLECTION
As mentioned earlier, following are insights related to my interests in the project: morals and ethics, digital identities, and systems (explainable AI).
MORALS
This project taught me that when it comes to decisions related to analysis and diagnosis, as with Healthy Privilege, sometimes slower is better. If there is not a true urgency, don’t create one. Quick decisions can be regretful or problematic ones.
Further, the project confronted me with the preexisting broad question: Should we be inserting AI in seemingly everything? I teased this out a bit and reframed it to: If this were a real scenario that was poised to go out into the world, would it be worse to somehow fail in the creation and deployment of this system, or not to attempt this type of system at all? After all aren’t we learning along the way? And so, perhaps a takeaway is about keeping the stakes high enough to be meaningful, but not so high as to cause undue irreversible harm?
DIGITAL IDENTITIES
This topic can be seen as one about control and access. While I enjoyed the research and making involved in this project, I find this panopticon like of system in Healthy Privilege to be very troubling. Problems range from the ideas around who owns your data, to what implications intended and otherwise it can have on all aspects of your life.This is due to the connected “partner programs” where we see the data continue to leak outward. We see glimpses of similar systems in the world today, and I find it troubling that there is not more concern about this “unauthorized or unintended data leakage”. This is also an area where I see opportunity as a designer to influence these concerns, especially as the scale and impact of AI exponentially increases.
SYSTEMS (explainable AI)
I playfully jest in the video that this AI is explainable since it reveals a portion of why it is making decisions. However this is false and problematic in its own right. As with any AI, I thought a bit about how data sets influence the outcomes. In this project I played with using reddit, Facebook, blog posts, daily news, etc. more heavily than medical journals or other reputable sources of medical information. This spun off into the following questions:
-How does AI deal with determining intention when the human may not be clear or decided on this yet? Are we doomed to always follow a statistical lead? Does this get back to the potentially homogenizing of humans power as can be explored with things like autocorrect etc.? These questions leave me curious about designing for improvisation without being reckless.
-If AI is not explainable, how might running simulations from which to optimize or tweak systems be helpful or problematic? We put our trust in the system, but what happens with the inevitable unexpected or random response? With this in mind, how might we design a system to maintain trust without being explainable? I think a case can be made that there are cases of this happening now, and wonder what consistencies may be found by studying successful cases.
DESIGN IMPLICATIONS
Responsibility of the designer is to be thoughtful and purposeful in how we guide AI development and implementation. If we don’t do this, we have failed. This was true in the past with design, but the sheer scale and potential reach and ripple effect of AI demands we consider impact on development of social structure and cultural values, that is technological determinism. Perhaps, as with chatbot design, we need to narrow scope and be honest about the limits of AI and the systems they live within.
MATERIALS AND TOOLS
-IBM Watson Assistant (chatbot backend)
-IBM Cloud Text-to-Speech and Speech-to-Text (chatbot frontend)
-Audio recorder
-Adobe Premiere
-Adobe Illustrator
-Adobe Photoshop
-Apple Keynote
-Omnigraffle
PROJECT BRIEF
What will it be like to co-exist with other intelligences?
Create a project that reflects this future world where people and AI systems co-exist and interact on an everyday, casual basis. Design a particular scenario and/or question and play it out in a detailed way to discover new things about AI and people.
How will people and ubiquitous smart things have relationships, collaborate, cooperate, understand each other, and generally co-exist? How will this interaction work as people and things get to know each other, come and go, and change? What will the diverse and multiple communities of people and other intelligent systems be like in the mundane, day-to-day future?
CLASS
Mundane Futures: Co-Existing with Other Intelligences
INSTRUCTOR
Phil van Allen
Knock-knock
OVERVIEW
Knock-knock is more than a smart doorbell, it is an intelligent collaborator. The system extracts data from publicly available resources, and the homeowners social networks. The resulting behaviors are based on data analysis, machine learning, and sentiment analysis. Knock-knock explores the benefits of human-machine collaboration, while prioritizing the owners privacy and time.
Unwanted solicitors and nosy neighbors can be disruptive and frustrating. Knock-knock provides a way to avoid undesirable interruptions, and simplify interactions with those you wish to allow inside your home. Natural language processing, along with built-in activity recognition and anomaly detection provide superior home gatekeeping.
Knock-knock from David Chan on Vimeo.
CONCEPT DEVELOPMENT
The research involved to develop the concept for this project involved the following:
-Uncovering the distinctions between voice vs. text chatbots
-Learning about Speech Synthesis Markup Language (for text to speech)
-Using a bot as router between humans
-Considerations around crafting personality of an AI
-Questioning the role of human intervention in this process
-Studying conversation management
-Deployment of a non-scientific version of sentiment analysis
With the decision to move forward with an intelligent doorbell system, three primary categories emerged in regard to anchoring the concept:
AI as Collaborative Intelligence
-Ability for personalized greetings and conversation
-System links to social media accounts to gather data and learn
-Users train system through interactions and with additional information
-User and system make decisions together based on personalized recommendations
Prioritization of Privacy
-Control your time with fewer unwanted interruptions
-Manage access by selecting from presets or customize
-Do not disturb (DND)
-Contacts only
-Allow all
-Set “do not allow” list
Security Update
-Package confirmation and alerts: delivery driver acknowledgement and alert of package receipt via photo, text, email, etc.
-Video: analyze what is seen
-Anomaly Detection: detect things/activities out of the ordinary
-Activity Recognition: interpret human movement
MAKING
Once the framework for a concept was in place, next steps for creating this project involved iterative prototyping and testing by jumping back and forth between two core areas of interaction:
1. Digital Interactions: I focused on playing out the concept with the IBM Watson (the chatbot) in order to refine conversational nuances. In order to get started with the Watson system I took an online course about building chatbots and participated in a classroom workshop. Later in the process, I created an interactive voice bot to act as Knock-knock.
2. Physical Interactions: I built out and tested doorbell scenarios in a real-world context in order to refine body and verbal language interaction. This helped in decision making of how to film scenarios to tell the story.
REFLECTION
I enjoyed the storytelling aspect of this project. I was free to imagine and crafting this concept as one which starts as a product pitch about a positive relationship between a product and its owner then takes a twist. The “personality” that Knock-knock forms turn this hopeful interaction into a confusing, potentially problematic and upsetting experience for the visitors interacting with the AI.
Early on in this project I thought a lot about what it means to have AI be a collaborator vs. servant. In this realm, I imagined a collaborator that goes a bit off the rails in the name of reaching the targeted objective. Research on this topic involved reading Machines of Loving Grace: The Quest for Common Ground Between Humans and Robots by John Markoff. This book helped me consider semantics when it says that artificial intelligence (AI) wants replace people with machines, and intelligence augmentation (IA) uses similar machine learning technologies to assist human, rather than replace them.
Upon reflection, the three areas of interest in my own design practice that carried through from early explorations to the final concept of this project are:
1. Systems Known/unknown participants and outcomes (especially as focused on a particular moment)
2. Digital Identities Considerations of privacy/security of data as well as the potential implications of this data crossing back and forth from digital to physical identity
3. AI Ethics This is an emerging topic at large in the world of technology, and there are many questions as to what the future holds. In this project, what does the power structure look like? Who are the stakeholders? What potential is there to disrupt or challenge existing power?
MATERIALS AND TOOLS
-IBM Watson Assistant (chatbot backend)
-IBM Cloud Text-to-Speech and Speech-to-Text (chatbot frontend)
-Canon EOS 5D DSLR camera and tripod
-Audio recorder
-Adobe Premiere
-Adobe Illustrator
-Adobe Photoshop
-Apple Keynote
-Omnigraffle
PROJECT BRIEF
Your task is to create a project that takes a critical perspective of AI/Machine Learning by designing something that on the surface seems plausible and sensible, but on deeper analysis is useless, absurd, or just really off in some revealing way.
The term “useless” can be interpreted in a range of ways, but your project must take a position and dig into the challenges, affordances, unforeseen side-effects, and potential failures of artificial intelligence and machine learning. Your project must be grounded in insights drawn from actual machine learning experiments, but please take risks, and have a sense of criticality and humor.
CLASS
Mundane Futures: Co-Existing with Other Intelligences
INSTRUCTOR
Phil van Allen
Emo:b
OVERVIEW
The emotional beacon that travels with you, when your friends can’t.
In collaboration with Mina Lee and Maxwell Chen
Emo:b from David Chan on Vimeo.
STORY
The context of our project starts out with a weary traveler. As many of us have experienced, traveling is something that is difficult for a variety of reasons: time-change, diet change, homesickness, and the list goes on. The protagonist in our story, Maxwell, is constantly traveling from place to place.
Feelings of loneliness and boredom are often byproducts of Maxwell’s nomadic lifestyle. Although there are platforms that allow for direct communication, it’s not the first choice for Maxwell while he’s traveling because:
-He doesn’t like texting with his phone all the time
-Some times phone calls are a bit too direct
-He is not necessarily looking to explicitly reach out for his emotional needs
-What Maxwell is looking for is something that allows for unobtrusive emoting and a more natural means of communicating to his friends and loved ones.
CONCEPT AND METHODOLOGY
Our group designed and developed a gesture sensing device held by traveller as way to reach out and stay connected.
Authentic local woven material was selected as the enclosure for hardware and small battery.
Often when one is bored, there is a physiological tie to fidgeting as an action. This gesture of spinning the device was researched as the first of what we envision to be many available gestures to express emotion with the Emo:b.
GESTURE
Slow spin - expresses the feeling of loneliness or boredom. Ties to the physiological relief to fidgeting.
OUTPUT
Upon gesturing, Emo:b sends out a message to pre-programmed friend(s) from the device.
(For a more complete look at an Emo:b scenario, please view the video at the top of this page)
PROCESS
Following are process pictures from the hardware and software testing, from proof of concept, to mapping sensor readings to gestures.
REFLECTION
Where did we end up?
Once we researched and validated the concept on the ground in Mexico City, we pushed toward the technical MVP. We got to a meaningful form and gesture as a comforting non-invasive form of communication. We made technical progress to get to data to trigger notifications and push back to the device. Next steps in next slide including a question regarding matching gesture and data.
What were roadblocks?
Network issues (ACCD dropping), technical learning curves (like having two devices on one account [star icon], getting IFTTT SMS to work with data threshold code)
NEXT STEPS
If we were to build a next iteration of Emo:b we would look into the following based on our testing and feedback:
-Stylize method of “messaging back” (HTML + LED)
-Compress physical form based on testing
-Add “start” button to trigger sending sensor data
-Install external LED for “return message” notification
-Challenge ourselves to develop new gestures based on additional emotions
-Further refine mapping of data to gestures
MATERIALS AND TOOLS
-Particle Photon board
-MP6050 accelerometer/gyroscope sensor
-Particle code IDE
-Thingspeak
-IFTTT (If This Then That)
-Maize husk pouch
PROJECT BRIEF
This first project of the class is about sensor networks feeding data collection in the context of the Internet of Things. It involves “dashboards”, constrained resources, and working with a distributed team between Los Angeles, CA, and Mexico City. Specifically, the technology for the concept are restricted to a Photon IoT board and compatible accelerometer/gyroscope sensor. This project should result in a technical prototype which would serve as the embodiment of the proposed design concept and could be used to test the assumptions of the teams regarding the technology used, its implications for the target audience, and intentions communicated.
CLASS
Creative Technology 3
INSTRUCTORS
Maxim Safioulline and Joshua Walton
Futurecasting
CAPSTONE PROJECT ABSTRACT
Futurecasting is a design fiction centered around new technology which allows people to create and store idealized versions of experiences called “made-moments”. The process itself is not unlike taking pictures with a smartphone. These made-moments are compiled and stored until a person’s death, at which time they feed artificial intelligence designed to outline and influence a roadmap for that person’s reincarnated life.
Content for this project comes from the following three sources which weave together to form the narrative. 1) Journal entries captured throughout one person’s engagement with the technology. 2) Information from the Futurecasting company such as account profiles, usage statistics, and terms of service. 3) Mass media, in the form of interviews and articles which look at power structures, government regulations, and the human condition.
This fiction is an opportunity to study two familiar concepts in a speculative context. First, the effect on individuals and societies of being so focused on a possible future, that they are no longer present in daily life. Second, this is an exploration into authenticity. What happens when people accumulate made-moments that do not align with their character?
Further, might this technology be providing a form of immortality through its system of self-curated reincarnation? This is valuable to consider as there are inevitable unintended consequences related to new technologies like Futurecasting. It is irresponsible to design and deploy these applications without first examining the permanent effect they may have on individuals, culture, and society.
PROJECT BRIEF
Design a speculative, critical, or fictional document that communicates a sense perception of time that lies outside the normative/dominant time-sense created by contemporary global capitalism. (It could be a marginalized time sense that exists under or in relation to global capitalism, for instance something relevant to a particular group of people or things.)
The form, material and format of the document are the result of your own design decisions and intent. Specificity is everything. Crucially, your completed project will demonstrate that you have made decisions about the following:
-What is the context in which your document was created, lives, and is used?
-What is it for?
-What is the form/format?
-Who is it for?
-How is it to be used?
-How does it engage design discourse?
CLASS
Critical Worldviews I
INSTRUCTOR
Elizabeth Chin
Crossing the Line:
POTUS Narrates Entitled Parking
OVERVIEW
We’ve all witnessed poor parking. But what’s really going on here? I dug into this issue and developed this project, Crossing the Line: POTUS Narrates Entitled Parking. What I discovered was that the bigger and more expensive a vehicle is, the more likely it is to park poorly. Further, increasing the threat of citation for poor parking has little impact on the bad behavior of the most blatant offenders, as these actions come from a sense of entitlement and self-importance rather than a lack of awareness for the rules. This project offers a non-violent and non-destructive outlet for the frustration of other drivers when the encounter poor parking.
When you encounter poor parking, simply tag and post a picture of the infraction to social media. The image is pulled into the parking grid and uses artificial intelligence to assign a Donald Trump quote to the image. Anyone can visit the parking grids and click to see what each "parking lot" contains. The example below uses the public parking lots and structures in Santa Monica.
WALKTHROUGH AND PROCESS
Roland Barthes "signifier, signified, sign" provided the framework used in order to connect the dots behind the concept of this project which resulted in the system at play. See the image below for more detail. I will go a step further and say that this entitled and self-important behavior has become more prevalent under the divisive and vocal POTUS. The creates an unfortunate feedback loop where bad behavior fuels bad behavior.
On the Technical Side:
-This demo represents data from the last 24 hours
-Users can filter to display totals for day week, month, year
-The pictures are pulled from social media (FB, Twitter, Instagram) via keywords and hashtags
-The lot or structure location is determined by looking at EXIF (Exchangeable image file) data which is embedded in the image to find the GPS coordinates
-The Trump quote selection is determined by analyzing any text that may accompany the picture and by using AI to determine specific parking characteristics and potential violations
REFLECTION
Out of the prototype that I built for this project came two main insights.
Insight #1 - Getting Meta
Again, this is about poor parking resulting from a sense of entitlement and narcissism. The pairing of Trump quotes with these images combines two different mediums with this common theme. More than just a pairing, there is a meta theme at play. As mentioned earlier, I posit that the presidents well documented poor behavior is having a negative impact on parking by emboldening this sense of self-importance and contributing to societal selfishness in many people.
Insight #2 - Location Matters
People in smaller parking lots in busy places such as Mid-City and Main Street lots are more likely to document poor parking out of frustration. For the same reasons there is a higher incidence of poor parking in these lots.
-They might be late for an appointment
-Have limited time on a lunch break
-Might be hangry for dinner
In contrast, people in large parking lots, such as the beach lots, are more likely to document poor parking out of amusement as there is more opportunity to park creatively in these large lots.
-People are there more often to have fun and tend to be more relaxed
-They are there longer term so there is less vehicle turnover vs. smaller lots
-This longer term commitment to longer term parking also results in taking more time to park properly or find a space that is appropriate for your vehicle
Further, people in parking lots are more likely to document because you can see more from one perspective and they are likely to walk past more cars exiting a lot than on different floors of a structure. Also, public parking structures in Santa Monica have cameras at every space capturing license plate info which does discourage poor parking.
In Closing
I began this project using my past Parking Systems project as a base. This was a very detail and data driven project that I wanted to make interactive in some way. It was very helpful in housing the idea behind this project and prototyping. As a next step, I would like to detach the idea from the grid map and move it to a pure social media experience with the aid of something like a twitter bot.
Prototype Demo
Example outputs
MATERIALS AND TOOLS
Pen, pencil, paper (sketches and rapid prototyping)
iPhone 5S (field video capture)
Adobe Photoshop
Adobe Illustrator
InVision (digital prototyping)
Media projector
PROJECT BRIEF
This final project is a synthesis of Dev Projects, Creative Technology and Media Design and is an opportunity for refinement and integration of your ideas and making. The specific theme is yours to choose, but you should combine some of your concepts, methods, ideas, and/or designs from all three classes, and create a new, more refined project by moving these forward. Rather than designing something overly complex or completely new, we recommend that you create something more contained, but highly refined in it’s technical, formal, and conceptual aspects. This project is about creating a refined and nuanced project that utilizes what you’ve learned and the interests you’ve developed in the three classes.
CLASS
Independent Project (combining the classes: Dev Projects, Creative Technology and Media Design)
INSTRUCTORS
Phil van Allen
Anne Burdick
Kevin Wingate
Shoe Memoirs
OVERVIEW
My design intention for the Shoe Memoirs project was to uncover the hidden stories people attach to the shoes on their feet. I did this by offering shoe portraits to people. To elevate the significance of the shoe portrait, I used a solid black cloth as a backdrop. To provide a space for others to learn about the interactions, I created a blog with the portraits and transcribed stories.
PRESENTATION
In order to capture shoe portraits, I added technology to a picture less frame. Participants look through the frame at their shoes, then press a button to take a photo. The image is uploaded to a Tumblr page dedicated to the images and stories for the project. Here are three of the top insights from testing prototypes during the Shoe Memoirs project.
1. IF YOU BUILD IT THEY WILL COME
The humble beginnings of the "Old and Inferior" rapid prototype can be seen in the images below. The plain looking frame with an iPad mounted inside served me well as a way to get into the field and test. After speaking with a stranger and listening to their shoe story, I discovered that I took an almost apologetic approach of my using the frame to capture images of shoes. Although the idea for what this project might be, and how it might evolve was in my head, there was still a lot of room to develop the design and approach.
In the "New and Improved" picture, the tech and design were iterated upon, deployments were more successful and the frame helped me tell the story of Shoe Memoirs instead of me trying to sell something invisible. Now instead of me confusing people by pulling the old frame out a bag at the end of the engagement, I was able to own it was free to lead with the frame if appropriate. Also, the people became more invested in the interaction as they were able to take their own shoe memoir photos.
2. LOCATION, LOCATION, LOCATION
I learned from deploying my prototype that the frame and backdrop seemed to either attract or repel people. This behavior appeared to be largely dependent on location.
Farmers Market
There were several instances at the farmers market where someone was engaging with the frame and peripheral curiosity drew people in. In the example image below, the woman was so eager to see what was happening and use the frame she interjected in the conversation. The man I was speaking with moved to the side and she sat next to his family to take a picture. The shoes of this wife and child can be seen in the bottom right corner.
Boardwalk
Later the same day, about a quarter mile away, I had a completely different experience. People were practically repelled by the frame and backdrop and actively avoided me. I had a similar experience at a nearby popular shopping area, though not quite as extreme.
Upon reflection, I believe this stark difference occurred because people come to the farmers market as a destination with a sense of community and expect to talk to other people, farmers, etc. At the boardwalk, there may be an acceptance that you will see other people, but not necessarily engage with them. There is more movement, and when people are stationary they are often skeptical of strangers approaching them.
3. OWN IT
I learned a lot through the different iterations and prototype deployments. These experiences helped inform my approach in the field. Deployments became more comfortable as I gained knowledge of how to talk about the frame and adapt on the fly to gain and maintain engagement. In the example below, I was waiting for my lunch, and saw an opportunity to engage with someone who was also waiting for food. These relaxed interventions were very rewarding, empowering, ultimately led to more interaction. For example, in the background of this picture we see another instance of curiosity.
TECHNICALLY SPEAKING
For this project I used Python to program a Raspberry Pi and control a camera module. The program is triggered by pressing the red button. This initiates a short countdown (blinking red LED), after which the photo is saved to a micro SD card and uploaded to my Tumblr blog (blinking blue LED). Upon completion, the green LED flashes to tell you the upload and save were successful. The frame is then ready to take the next picture. The hardware, along with a battery pack was mounted to the back of the frame. Holes were drilled out for the LEDs and the button.
ADDITIONAL INFORMATION
I realized upon reflection that Shoe Memoirs is no longer just about strangers. This is as much about conversation with someone you may know and capturing their shoe stories.
The Tumblr page for Shoe Memoirs can be found at: tumblr.com/blog/shoememoirs
MATERIALS AND TOOLS
Raspberry Pi (A+ and 0w)
Pi camera module
Python
Various wires, LEDs, buttons
Battery pack
Picture frame
Foam core
Black felt
PROJECT BRIEF
Conduct field research looking at the relationships surrounding humans and technology. Develop and evolve a design intention where you use technology to create, foster, enable and facilitate interaction. Pay close attention to the exchanges that result from or are instigated by these productions—engagements, happenings, provocations, experiences. These two threads are commingled to offer new models for understanding and more importantly exploring interaction through making/prototyping.
In progress project critiques will drive general discussion of terms, considerations, and the specifics of form as embodiment of idea. Discussions will include both the possibilities and limitations of your proposition as well as how the material and experiential choices embody the intent.
CLASS
Dev Projects 1
INSTRUCTORS
Elizabeth Chin and Elise Co