Knock-knock
OVERVIEW
Knock-knock is more than a smart doorbell, it is an intelligent collaborator. The system extracts data from publicly available resources, and the homeowners social networks. The resulting behaviors are based on data analysis, machine learning, and sentiment analysis. Knock-knock explores the benefits of human-machine collaboration, while prioritizing the owners privacy and time.
Unwanted solicitors and nosy neighbors can be disruptive and frustrating. Knock-knock provides a way to avoid undesirable interruptions, and simplify interactions with those you wish to allow inside your home. Natural language processing, along with built-in activity recognition and anomaly detection provide superior home gatekeeping.
Knock-knock from David Chan on Vimeo.
CONCEPT DEVELOPMENT
The research involved to develop the concept for this project involved the following:
-Uncovering the distinctions between voice vs. text chatbots
-Learning about Speech Synthesis Markup Language (for text to speech)
-Using a bot as router between humans
-Considerations around crafting personality of an AI
-Questioning the role of human intervention in this process
-Studying conversation management
-Deployment of a non-scientific version of sentiment analysis
With the decision to move forward with an intelligent doorbell system, three primary categories emerged in regard to anchoring the concept:
AI as Collaborative Intelligence
-Ability for personalized greetings and conversation
-System links to social media accounts to gather data and learn
-Users train system through interactions and with additional information
-User and system make decisions together based on personalized recommendations
Prioritization of Privacy
-Control your time with fewer unwanted interruptions
-Manage access by selecting from presets or customize
-Do not disturb (DND)
-Contacts only
-Allow all
-Set “do not allow” list
Security Update
-Package confirmation and alerts: delivery driver acknowledgement and alert of package receipt via photo, text, email, etc.
-Video: analyze what is seen
-Anomaly Detection: detect things/activities out of the ordinary
-Activity Recognition: interpret human movement
MAKING
Once the framework for a concept was in place, next steps for creating this project involved iterative prototyping and testing by jumping back and forth between two core areas of interaction:
1. Digital Interactions: I focused on playing out the concept with the IBM Watson (the chatbot) in order to refine conversational nuances. In order to get started with the Watson system I took an online course about building chatbots and participated in a classroom workshop. Later in the process, I created an interactive voice bot to act as Knock-knock.
2. Physical Interactions: I built out and tested doorbell scenarios in a real-world context in order to refine body and verbal language interaction. This helped in decision making of how to film scenarios to tell the story.
REFLECTION
I enjoyed the storytelling aspect of this project. I was free to imagine and crafting this concept as one which starts as a product pitch about a positive relationship between a product and its owner then takes a twist. The “personality” that Knock-knock forms turn this hopeful interaction into a confusing, potentially problematic and upsetting experience for the visitors interacting with the AI.
Early on in this project I thought a lot about what it means to have AI be a collaborator vs. servant. In this realm, I imagined a collaborator that goes a bit off the rails in the name of reaching the targeted objective. Research on this topic involved reading Machines of Loving Grace: The Quest for Common Ground Between Humans and Robots by John Markoff. This book helped me consider semantics when it says that artificial intelligence (AI) wants replace people with machines, and intelligence augmentation (IA) uses similar machine learning technologies to assist human, rather than replace them.
Upon reflection, the three areas of interest in my own design practice that carried through from early explorations to the final concept of this project are:
1. Systems Known/unknown participants and outcomes (especially as focused on a particular moment)
2. Digital Identities Considerations of privacy/security of data as well as the potential implications of this data crossing back and forth from digital to physical identity
3. AI Ethics This is an emerging topic at large in the world of technology, and there are many questions as to what the future holds. In this project, what does the power structure look like? Who are the stakeholders? What potential is there to disrupt or challenge existing power?
MATERIALS AND TOOLS
-IBM Watson Assistant (chatbot backend)
-IBM Cloud Text-to-Speech and Speech-to-Text (chatbot frontend)
-Canon EOS 5D DSLR camera and tripod
-Audio recorder
-Adobe Premiere
-Adobe Illustrator
-Adobe Photoshop
-Apple Keynote
-Omnigraffle
PROJECT BRIEF
Your task is to create a project that takes a critical perspective of AI/Machine Learning by designing something that on the surface seems plausible and sensible, but on deeper analysis is useless, absurd, or just really off in some revealing way.
The term “useless” can be interpreted in a range of ways, but your project must take a position and dig into the challenges, affordances, unforeseen side-effects, and potential failures of artificial intelligence and machine learning. Your project must be grounded in insights drawn from actual machine learning experiments, but please take risks, and have a sense of criticality and humor.
CLASS
Mundane Futures: Co-Existing with Other Intelligences
INSTRUCTOR
Phil van Allen