prototype | studio any

Daphne's Diary

Fri, 03 Nov 2017 12:56:19 +0300

Daphne’s Diary is a prototype for historical interactive location-based audio narrative Geocaching tours, developed during the COST Cyberparks Training School titled “Virtual Flâneur of Mediterranean Cities” that took place at the Cyprus Institute, Nicosia, during the week of 30.10-3.11.2017.

On February 2018, the Cyberparks COST Action awarded the project a short-term scientific mission (STSM) grant, for further development of the project with local cultural institutions in both sides of Nicosia, in collaboration with the Cyprus Institute. The STSM was conducted between March 22 and April 4, 2018.

Credits

Team members

Constantinos Miltiadis, Institute of Architecture and Media, TU Graz
Paul Schmidt, Media & Culture, Utrecht University
Aurimas Žemaitaitis, Kaunas University of Applied Sciences
Tamás Tóth, Széchenyi István University, Győr
Simonas Skarzauskas, Mykolas Romeris University, Vilnius

Text, story & documentation: Constantinos Miltiadis
Voice acting & audio recordings: Paul Schmidt
Website design: Tamás Tóth
Videos & editing: Aurimas Žemaitaitis & Paul Schmidt
Location scouting: Aurimas Žemaitaitis, Paul Schmidt & Simonas Skarzauskas

Acknowledgements

The team would like to thank George Artopoulos of The Cyprus Institute for organising the workshop and supporting the project, as well as Nikolas Bakirtzis of The Cyprus Institute, and Elena Poyiadji of the Leventis Municipal Museum for their support with historical research and material.

John Paul George & Me

Mon, 06 Apr 2015 18:11:20 +0300

John Paul George & Me is a collection of interactive wearable musical instruments for kids. Each T-shirt represents a specific instrument and all together form a rock ensemble. Through the seemingly magical clothes that produce sound by touch, the project brings to children an interactive musical game, encouraging and stimulating their musical interest as well as collaboration with each other to perform music. The project was developed by textile designer Selina Reiterer and computational designer Constantinos Miltiadis for exploring playful applications of smart materials and interactive wearable technology.

The project includes three T-shirts, a microcontroller with a Bluetooth Low Energy (BLE) module, and a mobile app.
The three individual T-shirts were inspired by and designed after the standard rock trio representing a guitar, a bass, and a drum set. Each shirt has a distinct pattern, based on the attributes of the instrument it represents, which each shape forming a uniform conductive region sensitive to touch. The patterns were silk-screen printed on plain T-shirts with electrically conductive paint, a flexible, washable and non-toxic chemical mixture.
In the back of each garment is a removable pocket with an Arduino microcontroller that receives touch signals from each shape through conductive thread.
The touch signals are then sent wirelessly via BLE to a mobile application which triggers different audio samples corresponding to each garment-instrument. Multiple T-shirts can be connected to a single mobile device, or each band member can use their own. The device can be connected with headphones and external speakers to amplify the sound output.

Additional information

“John Paul George & Me” was exhibited at the 2015 Maker Faire in Rome.
The project received the 2^nd prize in the YouReshape wearable technology competition (archived) in August 2015.
It was featured at the Materiability Research Network at John Paul George & Me (Materiability Research Network), and the Bare Conductive Blog at Bluetooth Connected Musical T-shirts (archived).

Credits

Video credits

Kids: Beda & Vasco
Filming: Anna Maragoudaki & Achilleas Xydis
Editing: Anna Maragoudaki
Music: Constantinos Miltiadis

Project credits

Selina Reiterer (textile design & material research)
Constantinos Miltiadis (electronics & app development).

Developed in 2015 at the Chair for Digital Architectonics (née Chair for CAAD), ETH Zurich.

ESUM: Anthropocentric Urban Sensing Unit

Fri, 06 Mar 2015 12:56:02 +0300

Design, development, and documentation of the ESUM backpack, a standalone mobile prototype for an ‘Anthropocentric Urban Sensing Unit’ at the Chair of Information Architecture ETH Zurich.

The ESUM project was initiated by Reinhard König at ETH Zurich. The concept behind the backpack sensor prototype was to develop a mobile sensor unit to measure, analyse and correlate both ambient and biometric data from a subject’s itinerary in an urban context, and eventually correlate environmental qualities with psychosomatic effects.

The ESUM backpack sensor prototype was developed and documented with the intention to be easily replicated by the different research labs associated with the project. It includes several off the shelf electronics components, a custom housing that can fit in a small backpack, and custom software for logging sensor values. The software was developed as a Visual Studio C# console application, and is responsible for synchronous data logging from the different sensor units with varying frequencies and temporal dependencies – sensor fusion and analysis was implemented via machine learning at a later stage of the project (see Ojha et al. 2019).¹ The ESUM case is made from plywood and plexiglas, and can be laser cut and assembled in less than 1 hour.

This work is part of ESUM-Analysing tradeoffs between the energy and social performance of urban morphologies, a joint project between the Chair of Information Architecture, ETH Zurich, Zurich, Switzerland and the Chair of Computer Science for Architecture, Bauhaus University Weimar; supported by the Swiss SNF and the German DFG funding programs.

ESUM backpack prototype

Prototype parts

Barebone PC
Battery unit
Emotiv Epoc+ 16 point wireless EEG
Libelium Meshlium WiFi sniffer
Empatica E4 biometric watch (electrodermal activity, photoplethysmography, peripheral temperature, motion)
GPS unit
Atmospheric pressure sensor
Temperature sensor
Humidity sensor
O3 sensor
Air pollutants sensor I (C4H10, CH3CH2OH, H2, CO, CH4)
Air pollutants sensor II (C6H5CH3, H2S, CH3CH2OH, NH3, H2)
CO2 sensor
NO2 sensor
Sound pressure sensor
Luminosity sensor
Dust sensor

ESUM project description²

In this research project, we investigate the impacts of urban morphology (UM) on citizen’s social potential as a function of accessibility and perception and compare it with the impacts of UM on building energy consumption for parallel case studies in Zürich Switzerland and Weimar Germany. This is of particular interest since urban planning decisions of urban morphology have long–term implications that affect not only the energy performance of the building stock but also on people’s experiential quality of the city environment.

In order to understand, measure and predict the impacts of UM on the perception of citizens we developed a three-tiered empirical study, which draws upon the participants’ experiences in a real-world experiment, along with two controlled experiments using a 360degree video and a gray-scaled 3D virtual reality (VR) model of the city. All three experiments are conducted using the same predefined path in each of the two cities, for a total of six participant based experiments. In each experiment, the participants subjectively rate the urban environment through surveys for an understanding of their explicit perception.

We also go beyond subjective impression ratings to gain further insight into the implicit perception of the participants by simultaneously measuring electro-dermal activity (EDA) data collected via wearable devices. The EDA data is processed to quantify the frequency, duration, and location of arousals along the experimental path to show a measure of excitation. The explicit perception data provides the sentiment of the experience such as spacious/narrow, beautiful/ugly, or private/public (among others) while the implicit perception provides the “raw” arousal-based perception from participants.

We also consider the impacts of urban environment features on perception, including temperature, illuminance, sound, and dust using data collected via a “sensor-backpack,” specifically designed for the use in the real-world participant studies. The data produced from this experiment is highly complex; the seven sensors have varying frequencies and varying temporal dependencies. We, therefore, developed an information fusion mechanism to combine the spatial-temporal data to produce a structured dataset, which is analyzed using four state-of-the-art Machine Learning algorithms. The quantified physiological responses or “human perception” data are paired with the sensor-based environmental measurements using Machine Learning based knowledge discovery approaches. These include classification, fuzzy rule-based inference, feature selection, and clustering to derive underlying relationships between participants’ physiological response and environmental conditions. The results indicate that the participant’s implicit responses are affected by both urban morphological features measured as participant field-of-view and street width as well as the dynamic environmental features temperature, noise levels, illuminance and traffic speed.

Ojha, Varun Kumar, Danielle Griego, Saskia Kuliga, Martin Bielik, Peter Buš, Charlotte Schaeben, Lukas Treyer, et al. ‘Machine Learning Approaches to Understand the Influence of Urban Environments on Human’s Physiological Response’. Information Sciences 474 (1 February 2019): 154–69. https://doi.org/10.1016/j.ins.2018.09.061. Project repository: ESUM-project. ↩︎
https://archive.arch.ethz.ch/esum/about.html#esum ↩︎

Shifting perspectives

Mon, 10 Nov 2014 11:03:03 +0300

Shifting perspectives is an interactive kinetic installation prototype for projection mapping developed in 2014 between Zurich and Delft with Demetris Shammas, Achilleas Xydis, and Mariana Popescu (as PUNCH Collective).

The installation is a kinetic 6×4 grid structure used as a canvas for interactive projection mapping. The structure consists of 24 square panels assembled with rotary joints, so that when pulled appart the panels rotate individually and the whole structure expands. The movement of the structure is controlled by stepper motors that pull the structure from 2 external points. The three custom software developed for the installation were: a Processing-Java program for real-time sound analysis (FFT); an Arduino-based software for controlling the stepper motors; and a Unity program for projection mapping, all synchronized with each other via OSC. Using real-time sound analysis the installation projected audio-responsive graphics mapped on the continuous movements of the grid structure.

Shifting perspectives was developed in Zurich, fabricated and assembled in Delft for the event 7 Sins, in November 2014.

project Anywhere

Fri, 07 Nov 2014 18:04:19 +0300

The ultimate display would, of course, be a room within which the computer can control the existence of matter.
(Ivan E. Sutherland, The Ultimate Display 1965)

project Anywhere is a framework and proof-of-concept prototype for full-body interactive wireless VR, developed at the Chair for CAAD, ETH Zurich in 2014, to investigate notions of virtual space and time, and to explore designing synthetic spatiotemporal sensory experiences. The project was initiated in mid-2014, following the release of the first commercial VR devices that featured limited bodily involvement and capacity for movement. Addressing these limitations, this project developed a VR framework consisting of a mobile application, custom wireless VR HMDs and data glove prototypes as well as skeleton tracking, to produce a vivid real-time interactive presence in a virtual reality environment In that, project Anywhere demonstrates the potential of virtual space as a valid architectural medium in itself (instead of its use as an intermediary form for design or visualization), that allows experimenting with synthetic kinaesthetic, visual and auditory sensory experiences. Concurrently it explores the capacity of designing spatiotemporal environments with an embedded dimension of time. The project’s title, “Anywhere”, refers to the generic nature of the digital-virtual environment that can be made to simulate any physical context: real or imaginary.

Video demo

Project description

The project consists of the following bespoke software and hardware prototypes:

a mobile multiplayer VR app,
Omnimask: a 3D printed mobile VR head-mount ,
Inteligloves: a pair of custom wireless data gloves, and
Omnitracker: a sensor fusion software.

The main component of the project is a custom mobile application (1) developed in Unity as a multiplayer VR videogame. Mounted on a VR mask (2), the app provides a stereoscopic viewport for each individual user, through the point of view of their avatar in the shared virtual environment.

Each user’s spatial presence is tracked in real-time by their head-mount, data gloves, and a Kinect skeleton tracking sensor. Cumulatively, head, gesture and skeleton tracking provide 86 degrees of freedom, fused together by the Omnitracker (4), a software written in Java/Processing. These data are synchronized with a web cloud and are use to animate the user’s avatar in real-time (see image13).¹ Thus, any movement of the subject in physical space, such as walking, or even head tilting and finger movement correlates 1:1 to movements of their avatar.

Hardware prototypes developed for this project include a 3D printed VR head-mount (2) and a pair of data gloves (3). The VR head-mount was designed for an iPhone 5, and featured adjustable inter-pupillary distance, and focal length.
The data gloves (3) were developed using Arduino microcontrollers. Each glove features an XBEE transceiver module, a 9 DoF sensor, and up to 6 flex sensors (for tracking individual finger movement). These were mounted on a lightweight elastic 3D printed glove base, easily adjustable for different hand sizes. The framework was programmed to recognize hand gestures, used for performing various contextual virtual actions (see for example image 8), proving that a virtual presence can also be an active one.

Constantinos Miltiadis (2016). Project Anywhere: An Interface for Virtual Architecture. IJAC 14 (4).
Constantinos Miltiadis (2015). Virtual Architecture in a Real-time, Interactive, Augmented Reality Environment - project Anywhere and the potential of Architecture in the age of the Virtual. eCAADe 2015. TU Delft.
Photon Engine Dev Story (2015). Project Anywhere – Digital Route to an Out-of-Body Experience

Awards

Ivan Petrovic Prize for the “best presentation by a young researcher” for the paper “Virtual Architecture in a Real-time, Interactive, Augmented Reality Environment”. eCAADe 2015, TU Vienna, September 2015.
Second place Zeiss VR One Contest for Mobile VR Apps, 2015.
Arthur C. Clarke prize for “the most creative and unorthodox approach”. Museum of Science Fiction international architectural competition, 2014.

Press

The Guardian & The Observer Tech Monthly (2015)
Project Anywhere: digital route to an out-of-body experience : Constantinos Miltiadis’s project combines inteligloves and Kinect sensors to an Oculus Rift-style headset to provide the ultimate in impressive digital experiences.
Euronews High Tech; Euronews Next (2015)
Euronews Next. Project Anywhere: an out-of-body experience of a new kind Proyecto Anywhere, una nueva forma de jugar en línea - hi-tech [Spanish]
Reuters (2015)
Project Anywhere takes virtual reality gaming to new level
DesignBoom (2014)
Subjects manipulate project anywhere’s virtual universe in real-time
Museum of Science Fiction (2014)
The preview museum: A departure from museums as usual…
Fubiz (2014) Virtual Universe in Real-time [French]

Exhibitions & installations

Demonstration at the Carl Zeiss Headquarters, in the context of the Zeiss VR One App Contest. Oberkochen, Germany, June 2015
Public installation at TEDxNTUA. Athens, 17.01.2015

Project information

project Anywhere was developed as a postgraduate master thesis in 2014 at the Chair for CAAD, ETH Zurich, Prof. Ludger Hovestadt. It was first demonstrated at the Chair for CAAD in November 2014.

Credits

Concept and development
Constantinos Miltiadis
Prototyping support
Demetris Shammas & Achilleas Xydis

Video credits

Filming & Editing
Demetris Shammas
Photo documentation
Achilleas Xydis
Music
Michalis Shammas Subject
Anna Maragkoudaki

Images

For a discussion of the networking topology of this project see the 2015 Photon Engine Dev story: Project Anywhere – Digital Route to an Out-of-Body Experience. ↩︎

Vector Raster Bleep

Fri, 07 Mar 2014 17:15:18 +0300

VectorRasterBleep is an interactive installation developed as a playful interpretation of a baroque garden to the digital-virtual domain. The garden is conceived as the Queen’s antidote to boredom, a playground open to stimuli for subverting the normality of everyday life, and a gateway to the fantastic and phantasmic. Wearing an electronic suit, the Queen escapes to the garden to explore its wonders. Its physical dimension is a space –any generic space– tracked by infrared sensors and populated by machinery. Its virtual counterpart, however, is a live phantom that transforms continuously to reflect the Queen’s desires while she strolls the generic space.

Technical information:

Wearable suit with 20 LEDs, a speaker, and an Arduino microcontroller with an XBEE wireless module
Kinect motion tracking sensor
Custom software for motion tracking and interactive graphics generation developed in Java
Projector

Developed in 2014 for a course taught by Mihye An at the Chair for CAAD, ETH Zurich.

Interactive Kinetic Structures

Wed, 10 Oct 2012 11:13:53 +0300

Interactive Kinetic Structures is a prototype for an interactive architectural structure developed in 2012 as a diploma thesis project at NTU-Athens. The aim of the project was the production of a proof-of-concept for an architectural structure able to interact and reconfigure itself according to parameters of its immediate environment and different use scenarios. The concept for this work departed from the design flexibility of parametric design methods, and the intention of maintaining such flexibility as responsiveness even after the course of the design process:

What if we could transform the object of architecture from the design of the defined to the design of the unpredictable?

Video demo

Project description

The main component of the project is a custom software written in the Processing programming language. The software was responsible for the computational modeling of the structure, real-time interaction, and visualization (see video demo). The project also includes a functional physical modeling prototype with stepper motors and custom extendable beams developed with Arduino-based electronics, and controlled in real-time by the Processing software (see image 9). Moreover, a parametric CAD model developed in Rhino and Grasshopper was responsible for generating 3D models of instances of the structure using data forwarded by the Processing software via OSC. These were used to generate 3D renderings and 3D prints.

The structure itself consists of two parallel grids: a triangular grid at the bottom and a hexagonal grid at the top. The two grids comprise of extendable beams. These are connected to each other with fixed-length beams, while all joints allow for free rotation. The structure is supported by vertical extendable beams at the outline of the lower grid. This configuration makes up a rigid spaceframe which can be deformed while maintaining structural integrity by altering the beam lengths of the two parallel grids (see image 3). The structural deformation solver was modeled using a mathematical equation based on triple-sphere intersection. The structural detailing of the project provisioned for pneumatic extendable beams, and explored three types of joints using antiprism nodes with universal mechanical joints, ball joints, and solid elastic polymers (see project booklet).

The project investigated two distinct use scenarios: playful interaction with people and adaptation to environmental conditions.
The first scenario produced continuous animations and transformations the structure according to the movements and densities of people under it in real-time, thus in a way encourage playful interaction between people in public space (see image 5). This was modeled virtually using a boid system, and was demonstrated live using a Kinect (spatial scanning) sensor.
The second made use of a solar radiation model that would calculate the position of the sun for a particular geolocation so that the structure would adapt accordingly to generate the maximum possible area of shade at any given moment of time (see image 6).
Moreover, the project explored a range of possible configurations of the structure based on various mathematical equations such as hyperbolic paraboloid and monkey saddle surfaces as well as various trigonometric equations (see 3D renderings below).

Project information

Interactive Kinetic Structures was developed in 2012 as a diploma thesis project at NTU-Athens, under the supervision of Prof. Dimitris Papalexopoulos. It was presented to a four-memeber jury in October 2012 and graded 10/10.

The project was one of the finalists of the international competition ALGODeQ: Algorithmic Design Quest in 2013.

A paper based on this project was presented at the 2013 eCAADE conference at TU Delft, in September 2013.

Credits

Concept & development
Constantinos Miltiadis
Supervision
Dimitris Papalexopoulos
Additional support
Petros Koutsolambros, Ero Papavasiliou, Spyros Efthymiou, Eliza Neophytou, Antonis Tzortzis, Serafeim Matzoufas, Antreas Alygizos

prototype | studio any

Daphne's Diary

Credits

Team members

Acknowledgements

John Paul George & Me

Additional information

Credits

ESUM: Anthropocentric Urban Sensing Unit

ESUM backpack prototype

Prototype parts

ESUM project description2

Shifting perspectives

project Anywhere

Video demo

Project description

Related publications

Awards

Press

Exhibitions & installations

Project information

Credits

Video credits

Images

Vector Raster Bleep

Interactive Kinetic Structures

Video demo

Project description

Project information

Credits

Images

ESUM project description²