Goals, Concept

I was really fascinated by the examples of artificial agents we worked through in the Nature of Code class; there is just something incredibly fun about being able to create pseudo-lifelike behaviour in software.

My goal for this project was to improve my understanding of Reynolds’ steering behaviours and genetic algorithms by designing a “landscape” inhabited by different species of creatures. In this landscape the creatures seek food, feed, flee from danger, reproduce, age and die.  (The full proposal for the project is over here).

The Landscape

The landscape inhabited by the creatures contains a number of food-sources and threats.

Food-sources are indicated by green circles, and as the creatures deplete these food sources they shrink, disappear and then regrow slowly.

Threats are indicated by smaller red circles. Unlike food sources, these threats never deplete. If a creature comes too close, they are injured - proximity to the threat reduces the amount of energy that a creature has.

The creatures retain a memory of food sources and threats; these are indicated by the green and red icons that are displayed next to each creature. (This visual indication of creatures’ memory was one of the most useful things I made for the project - much more useful and immediate than print-line statements.)

The Creatures

The creatures have attributes and exhibit behaviours that help them survive:

• Energy/Life: The creatures are created with a finite amount of energy that is depleted when they move and over time. To replenish their energy supplies they must consume food.
• Seeking: The creatures will wander around the landscape at random until they find an object that interests them; at which point they will move towards that object.
• Vision: When the creatures are within a short distance of an object they can tell if it is food or a threat.
• Fleeing: The creature can move quickly away from threats.
• Feeding: The creature can arrive at a food source and consume it; the food source is depleted and the creature’s energy level increases.
• Reproduction: If the creature has sufficient energy and is close to another creature of its species, it has a chance of reproducing.
• Memory: Each creature can remember the food sources and threats it encounters in the landscape.

All of these behaviours are triggered depending on scenario the creature finds itself in; if it’s near an object, it will move towards it to examine it more closely and then feed/flee depending on what that object is. If the creature is near a potential mate, then a fitness algorithm will decide whether or not a new creature should be created etc.

Video

This video shows the version of the project I presented in class, with a couple of small tweaks. In the video you’ll see three species searching for food and encountering threats. From time to time two creatures of the same species will encounter one another and a new creature will be produced. To make the new creatures easier to see a grey circle is drawn around them for the first 10 seconds of their life. A higher resolution version of the video can be downloaded from Vimeo.

Further Improvements

There are lots of ways in which this project could be improved upon. I will definitely improve the genetic algorithm that is used when new creatures are created. At the moment there is no mutation, so the results are predictable and the fitness of the creatures doesn’t improve across generations.

As I said in my original proposal, I think it would also be great to tie some aspects of the landscape to information from the real world. Creatures could behave differently depending on the time of day, or they could respond to weather conditions. Food sources could regrow more quickly depending on weather etc.

Documentation

All of the code from the project is available on github; just clone it to your computer and then open the sketch in Processing. The code is reasonably well commented. Screenshots are available on Flickr and there are several work-in-progress videos on Vimeo.

You can download a much higher quality version of the video over on Vimeo. The code for all the experiments and projects shown in the video is available over on github.

We have a long way to go before we can start a project knowing that we’ll be able to focus on the needs of people and not on overcoming obstacles.

I’ve had a lot of fun lately programming little creatures to display life-like behaviours like finding food, wandering, fleeing threats. There is just something weirdly compelling about tweaking and shaping the way the creatures behave.

I’m going to continue this work. I’ll create a landscape populated by a number of different species of creature who exhibit different behaviours. They’ll be born, search for food, follow each other, flee from each other, seek shelter, reproduce and die. I’d like the creatures to have some sense of time and memory - things will happen depending on their age, and they’ll be able to store information about their interactions. This information will inform their decision making.

If I get time I may tie some of the attributes of the world to data from the real world; maybe some real weather data to inform weather in the world. Having any sort of direct communication between our world and the world of the creatures isn’t something I’m certain that I want to pursue; we’ll see.

The Creatures

The creatures will have some basic capabilities; abilities to walk, run, consume food, follow others. 

The creatures will want for things, they’ll seek food, find a mate, flee from predators.

They creatures will organise themselves into groups based on their attributes.

The passage of time will influence the creatures’ behaviour; during the day they will range far and wide energetically. At night they will seek shelter and stay close to one another.

Landscape

The creatures will roam around on a landscape, probably an island of some sort - a space with natural boundaries.

The island will have some natural textures or types of terrain and the creatures will interact with these textures - feeding on them, avoid them etc.

On Monday I presented the project to my Thesis classmates at ITP. I essentially summarised the background to the project, the different phases I had worked through and the results to date. The response was very positive. I’m proud of where I’ve managed to get to in 10 weeks, and I’ve learned a huge amount about my own design process.

Next Steps

I’m resisting the urge to add new things to the web application, and instead I’m focusing on making the interface a little more intuitive - and generally tidying up some bits and pieces.

On Tuesday of next week I present the project to senior staff at Memorial Sloan-Kettering for feedback and discussion. I’ll document what they have to say, and will try to implement any suggestions they have that can be done within the week. Anything that can’t be done within the week will have to persist into a subsequent project or phase, or taken as a requirement/idea for work they do in the future.

In this first experiment I wanted to create a “creature” that would wander an area. If the creature smells something, it goes to investigate. Once it gets within visual range it approaches cautiously. Once it gets very close and can see what exactly the “something” is it can make a decision to eat or run away.

I’m about half way through with this - the creature is wandering, sensing a far off object and going to investigate. I have to implement the cautious approach and fleeing.

The code for this simple experiment is on GitHub and is almost entirely based on Dan Shiffman’s examples of Reynolds steering behaviours in Processing.

Experiment 2 - Fleeing, Memory

In this next version the creatures are roaming a habitat. If they get within smelling distance of an interesting object, they go to investigate. If the object is a threat they run away quickly, retaining a memory of that object. The next time they come within smelling distance of that object they flee without going to take a closer look. You’ll see in the image that some creatures roaming the habitat have a little red circle in their “memory” denoting knowledge of a threatening object.

Next, I’ll introduce a second type of interesting object - food - that the creatures can feed upon, deplete and then leave. It might also be interesting to have the creatures share knowledge of threates with one another - but that’ll have to wait for version 4. I’m riffing on this for the rest of the day I think

Experiment 3 - Depleting a Foodsource

In this experiment the creatures are gravitating to a foodsource. As they feed, the foodsource is depleted. The more creatures who arrive to feed, the faster the foodsource is depleted.

When the foodsource is depleted, the creatures continue to wander.

A web application is not a service. Even though I’ve built this with real people in mind, people I met and interviewed during my research, I still have to put in place other things that would be necessary for this to be adopted as a real service - a timeline, coaching/training material for staff, help documentation etc.

I also have to establish exactly what hurdles exist that would need to be overcome before the service would launch - such as further testing, a security audit etc. At the end of the day it’s highly likely that for this to be used in a real context, with patients, an experienced developer would need to help out - but that is perfectly fine as far as I’m concerned.

So, I’m moving into a final phase - finishing out the look and feel of the web application, making supporting materials and writing up.

And like that, it’ll be Summer.

As I’ve been developing I’ve been testing the app on myself, and now two other volunteers - two members of staff from MSKCC have been wearing FitBits for the last week. They subscribed to the Activity App allowing administrators to see their progress, offer feedback etc. Having real data coming in has been incredibly useful.

In other news, I’ve been working with the hospital to see how this type of service could be useful after the end of my thesis - there are positive signs that my project could be included in a clinical protocol (the set of rules, procedures that govern a clinical trial on a particular patient population) which would allow me to roll it out to some bone marrow transplant recipients, gather data and then improve the system.

The next major milestone for me is the 4th of April when I present the working prototype of the service and supporting materials to medical and administrative staff at the hospital. After this milestone I’ll be moving into a tweaks phase and working almost exclusively on documentation, forward-planning.

Aside from being sick this week, I also started to feel a bit jaded by the grind of the solo, semester-long project. Working on solo projects is usually right up my street, but I’ve been too distant from my friends at ITP, and this has added to the normal challenges of working alone. At the half-way point of the semester I have time to adjust and re-connect; I just need to make sure that I do.

A tricky week.

The show runs from the 14th of March to the 18th of April and will also feature work by Jer Thorp, Marius Watz and others. I am looking forward to it.

Right now my focus is on building a simple web application that will allow patients to get a sense of their physical activity and monitor modest goals, while giving clinicians some oversight. I’m building the application using Sinatra. I’ve had some great help from Steven Klise, and I am slowly - ever so slowly - making progress. I’m successfully pulling data in from the FitBit API and storing it against a local user record. This is all I really wanted to achieve in the first few weeks. I now need to move on and implement the more specific pieces of the puzzle - a persuasive feedback system that draws clinicians and patients together with positive effect.

As I work on this phase I’ve been getting drawing on the insight from the research phase to keep me on track, and I also had the pleasure of interviewing one of the hospital’s patients this week. We talked about his treatment experience and about the challenge of maintaining physical condition, and he had some encouraging feedback on my project.

The next couple of weeks are really crucial; there’s so much to do - but I am feeling pretty positive (if a tad overwhelmed).

Once I had a simple test dataset I started to build the usual internal machinery of a sketch like this - functions to get the data and parse it into objects. I’ve had very little time to make the visual rendering step anything more than a Box2D hello world.

At the moment the processing sketch is loading information from a database. For each row of the data set an object is created and a corresponding image is loaded into it. Each image is analyzed to find its size and average colour. This information is used to inform the creation of Box2D elements, and then the object is placed into the Box2D world. The objects appear and are directed down with the force of gravity, coming to rest on a surface.

It’s super simple for now. I will probably not be using Box2D after I hand in the mid-term; I’m interested in getting away from the literal representation of the pieces of ephemera - when I have more time I’ll decide exactly what that means. I’m drawn to the idea that this junk is like leaf litter, or layers of time.

We shall see.

I’m building a visualization of ephemera and little items I’ve deliberately or inadvertently gathered in the last few years. I am quite a long way away from thinking about aesthetics, but this stuff resonates with me and may inform the rest of the project.

Accretion

Karamari Damacy - The quite bonkers game where you control a giant rolling ball that collects things as it moves.

Planetary accretion - In astrophysics, accretion is the growth of a body through the gravitation attraction of other matter.

Geology

image source

Detritus

image source

Flocking, Group Behaviours

image source

The first theme related to the the complexity and volume of information that patients and their families receive and understand during the bone marrow transplantation process. I saw how information offered to patients could seem overwhelming given their context, and I had some ideas as to how information could be delivered/experienced in a more natural way.

The second theme centred on patient’s level of physical activity. During my interviews with them, doctors and therapists had stated the clinical benefits of exercise before, during and after a bone marrow transplant. The reasons for exercise change throughout the patient’s journey, and the amount they need to do varies as well. In the patients’ context where they experience periods of isolation and some debilitating symptoms, encouraging activity is not straightforward. I had some ideas about the design of a persuasive system to encourage activity in patients.

I’ve decided to pursue this second idea for the remainder of my thesis. I’ll prototype a service that will encourage patients to remain physically active throughout their treatment in a way that is sensitive to their context. The service will also give their medical team some oversight, and will encourage them to play a role in motivating patients. Most of the key parts of this service will be non-technical - they’ll be tools/guides that will help to guide different members of the clinical team as to their role in deliver the service. Having said that I will build a web application of some kind to help act as a hub for the service. This project dovetails nicely with the work I did last semester on the design of persuasive health applications in my DIY Health class with Steven Dean.

I have written more detailed documentation around this concept; a project mandate, some constraints and design principles. This documentation will be my guide in the next four weeks of prototyping. I’ll also continue to work closely with the staff at MSKCC, and I’ll have a more formal checkpoint with them in late March.

In the next two weeks I’ll interview some patients who are currently undergoing treatment and these interviews will focus specifically on the challenges around physical activity. I’ll then complete a simple sketch/prototype of the service and gather some feedback.

So, things are going to some sort of plan. I hope you’re well.

Like a lot of people I seem to accumulate a lot of little items of paper and card; receipts, fliers, bills, menus. I keep some items deliberately because they represent memorable events, but most of it just accumulates - I’ve come to think of this as a form of inadvertent record keeping.

For my Nature of Code project I’m going to find every item of this ephemera I can find in my home. I’m going to scan each item, creating an image, and then store this image and any useful information I know about the item in a database.

I’ll feed this data into Processing, and represent the data using computer graphics and sound. I would like the items to have aesthetic qualities and behaviours related to their origin - so I’ll use data related to the item and a physics engine to power these qualities e.g. each new item finds other similar items, or flocks with items from the same time period.

My goal for the mid-term project is to find the physical items, scan and catalogue a subset of them, pull data into processing and represent them in some simplistic way. Over time I’ll add more complexity.

I’ll write more about the project over the next few weeks.

When I felt that I had a good handle on the background to bone marrow transplantation, and knew enough to start asking stupid questions - I moved into some primary research; interviewing people who work in areas related to bone marrow transplantation (with a focus on rehabilitation and/or post-transplant care). All of these interviews have now been transcribed, and I’ve drawn out key pieces of information and quotes from the transcripts.

I enjoyed the research phase a lot; I met some really remarkable people and had the opportunity to reflect on my design process in a lot of detail. I’ve identified a couple of exciting, practical projects that I think flow naturally from the research findings - so now I just need to pick what sub-set of these challenges I want to focus on for the next few weeks and get building.

Next week, I’ll complete my round of interviews and will spend some time doing observation exercises at the hospital. After that I get into concept development - then prototyping.

Time is flying, but I’m very happy with the work completed so far. 

A robot wandering in a seemingly random direction - noisy rather than random.

Wandering robots who keep track of each other.

Wandering robots who keep track of each other and are affected by an attractive force emanating from a dense body.

A number of objects orbiting a dense body.

A number of objects orbiting a dense body, leaving trails.

Next Steps

I want to start connecting some live data to the behaviour of these little systems - I’ll keep you posted.

While at ITP I’ve had an opportunity to explore some themes that are deeply personal and relevant to me, including health and well-being. My family has a longstanding with cancer and I’ve used some of my projects at ITP as vehicles to understand the disease.

Last year I completed a project called We Are Working On It - a visualisation of the many millions of scientific journal articles that have been written about cancer since 1980, presented alongside interviews with Ciara O’Driscoll and Laura Eccles; two research scientists at Memorial Sloan-Kettering’s Powell Lab. In my DIY Health class with Steven Dean we spent a full semester working to research and design persuasive systems to improve health; my final project for that class was Pilot, a device to monitor, analyse and improve swimming technique.

Collaborating with Sloan-Kettering

During one DIY Health class Steven Dean mentioned that Memorial Sloan-Kettering Cancer Center (MSKCC) had an innovation team whose job it was to design and improve the patient experience. This sounded really interesting, so I asked him to put me in touch with them.

I met with the innovation team at MSKCC several times in the Autumn and Winter of 2011. They were incredibly open and welcoming to the idea of a collaboration. We talked about possible areas of focus, and identified the bone marrow transplantation process as of particular interest. This is a hugely challenging form of cancer therapy that pushes the human body to its limit - the team were interested in doing whatever they could to improve the experience of bone marrow transplantation, especially the long period of rehabilitation and recovery. We agreed that this would be my (broadly defined) area of focus.

I wrote a detailed project proposal and put it to MSKCC in December of 2011. The proposal broke the project down into four phases; design research, prototyping, production and presentation. I’d work closely with MSKCC to understand the bone marrow transplantation process in detail (through interviews and desk research). Based on this work I’d identify a number of insights and use them to define some sort of intervention in the experience of patients. The rest of the project would involve making this intervention real, testing it and presenting it/gathering feedback.

I got the go-ahead for the project in mid-January of 2012, and for the last two weeks have been working through the start of my research.

Weeks 1 & 2: Crash Course

Since getting the go-ahead on the project I have really just had to dive into this world. I’ve done a good deal of reading on the bone marrow transplantation (BMT); focusing on academic literature that describes the process and support material given to patients and their families to explain the disease. I’ve done further reading on the process of rehabilitation and its many potential pitfalls. I’ve looked at the psycho-social aspects of the process and the roles that different people play in supporting the patient.

The reading has allowed me to understand at a very basic level what bone marrow transplantation is, when its necessary, who is involved in the process and how recovery and rehabilitation work.

I’ve also read documentation given to me by the MSKCC innovation team related to previous projects; they have worked with teams of outside designers in the recent past to develop concepts related to bone marrow transplantation. The work is pretty exceptional and the documentation has been immensely helpful, especially interview transcripts with medical staff, patients and families.

As well as desk research, this week I was able to tour the bone marrow transplantation ward at MSKCC with a member of the innovation team. We walked through the space to see where key steps of the process happen, and I was able to take photos and an audio recording documenting what I saw.

Next Steps

Next week I really want to take all that I’ve learned in the last few weeks and make it more tangible/concise; I’ll take my written notes and start to boil them down into little nuggets of information, key insights and quotes. Now that I have a grounding in the basics I can start to talk to people involved in the process; so next week I’ll interview members of the innovation team and medical staff. 

I’ve allowed for two more full weeks of research before I move on to concept development and prototyping; not a lot of time, but hopefully enough to inform the rest of the project.

Concept

In our Recurrning Concepts in Art class we were asked to take a previous project made at ITP and re-conceptualise/remake it without using technology.

Lily Szajnberg and I worked together to take two projects that involved tactility, physical sensation and strong emotion - merging them into one project that could serve as an exploration of Spectacle; in which we both shared an interest.

Our goal was to create an immersive, overwhelming experience for one person without using the traditional technologies of Spectacle.

Construction

We set out to create an environment that could physically envelope a participant. The key technical challenge was building a space big enough for a human. We investigated using sewer pipe and telephone kiosks - eventually we decided to build the “vessel” for ourselves.  We constructed a 4 feet x 4 feet by 6 feet “vessel” and used steel cables and rigging to suspend it from the lighting grid at ITP.

Originally, we thought that making this an automated/self-directed experience made sense - but we were keen to avoid complex technologies, sensors etc. The more we thought about having to interact with the participant directly, the more we liked the idea - the experience would remain personal and individual, but be directed from outside; this seemed appropriate. We added a layer of inflatable seating/bedding to the inside of the vessel, inflated by two high-powered electric air pumps.

Forever

Forever is an interactive art installation. The participant is asked to put on ear-protection and dark glasses. The participant enters a large vessel which is suspended from the ceiling. Two operators use pumps to inflate the interior walls of the vessel around the participant. After a period of time the walls are deflated and the participant exits. The next person is made ready, and enters the vessel.

Response

We were asked to reflect on the project. We chose to make an audio recording of our thoughts about the project. The recording was made on the 14th November 2011, one month before the ITP Winter Show.

  Forever by paulmmay

Reaction

Forever was presented for the two days of the ITP Winter Show 2011. We estimate that over 100 people took part in the experience.

Thanks

We’d like to thank our amazing classmates who inspired and challenged us throughout the semester, and Georgia Krantz for leading the Recurring Concepts in Art Class with great skill and dedication.

The Brief

The brief for the final project of our DIY Health class was to:

Design and develop an interactive system that helps you take stock of yourself to set goals and act upon your health and lifestyle. Your design intervention will include components that help you develop a systematic and ongoing analysis of how you are progressing with a keen focus on understanding and designing for closed feedback loops.

The other interesting part of the final project is that we were actively discouraged from working within our technical abilities - we were asked not to try to build these devices, and to focus on the needs of the user, the concept, the experience. This is really interesting in the context of ITP where it’s very often build first, ask questions later (to the detriment of the quality of the projects in some cases).

The Need

I started to run, swim and cycle late in life; when I was 26 I was asked to take part in a charity triathlon and I’ve dipped in and out of the sport ever since. It’s had a pretty big impact on my life; it’s weird to discover that you enjoy sport having actively shunned it for your entire life. So, I take a lot of pleasure from these three sports - and I want to get better at them for as long as I can. I suppose this is the core of my project.

When I took up sport I experimented with gadgets and devices that would help motivate me and get the most from my training. I’ve been disappointed by pretty much all of these devices. If physical activity is empowering, energetic, imperfect and sometimes emotional - the best most devices can do is quantification of time and distance. There is just this weird mis-match at an interface level between physical activity and sports devices.

So, this is where I started from - a desire to get better at physical activity and a dissatisfaction with simplistic, quantitative exercise devices. I decided to design a device that could help improve my capabilities by focusing on form rather than time/distance. Of the sports I do, swimming has the least to do with simple time/distance metrics and most to do with form - there just seems to be a need for a device that helps swimmers improve by focusing on smoothness.

Inspiration

I was really keen to avoid the problems that I see with other sports devices; the reduction of physicality to raw numbers. Instead, I wanted this device to feel more like a coach, a companion. I wanted the feedback to feel natural, appropriate to physical effort and water. I found a piece of writing by Jacques Cousteau about the invention of the aqualung that really resonated with me

From this day forward we would swim across miles of country no man had known, free and level, with our flesh feeling with the fish scales know.

The aqualung was a device that allowed us to thrive in an alien environment, augmented our capabilities and enlivened our senses. I wanted my device to unlock a sense of physical potential in the moment. I gave the project the name “Pilot” after the pilot fish - types of small fish that swim alongside sharks, mirroring their movements, sheltering from predators. There was just something about the relationship between the pilot fish and the shark that seemed to fit.

Prototyping

Building on the work we’ve done in class on feedback loops and cybernetic systems, I designed a system that could offer appropriate feedback to a swimmer in the water. I spent a lot of time crafting the description of the system; the story (which would eventually become the narration script for the video), the context of the system in a day and a series of storyboards.

As I was refining the description of Pilot, I was also refining how it would look and feel. I made several foam models that were small enough to fit in the small of my back, or comfortably on my chest.

One key thing I learned by making the models and simple holding them onto my body was that wearing a device in the middle of my body would result in all the feedback emanating from that one position. If I wanted to device to give feedback about my form in the water, then it’d be useful to have a feeling of stereo separation. As well as the central device I’d add two little pods out at the sides of my body so that I could receive more specific feedback e.g. feedback on the side of my body where I am not rolling effectively.

I took all this feedback and used a 3D printer to make the central computer and pods that you see in the final video. The printed pieces were mounted on some athletic elastic, similar to the type of thing you find in a heart rate monitor.

Shooting the Video

We were encouraged to produce a video as our final artifact for the project. I spent two hours in the endless pool at Swim, Bike, Run in Manhattan who were kind enough to let me use their pool. I shot slow motion footage of me swimming with and without the device using a Canon 7D. I used a housing for the Lumix LX5 to shoot underwater shots.

I’m very happy with the final video; it does a good job of explaining the thought behind Pilot and how it would be used in a short space of time.

Feedback and Next Steps

I’m really encouraged by the feedback I’ve had on the Pilot concept. The design of the system seems to work well in the design context, and there seems to be a genuine need out there for this device.
I’ve done some early investigation into what would be required to build the system for real, and I’m confident that it’s possible. I may tackle this as a project in 2012.

There has also been some (very welcome) negative feedback; that the device is too logical and that I didn’t take the opportunity that the brief offered to think in a purely speculative fashion. I agree with this, completely - and were I to do the project again I think I’d be more confident to push myself and the concept into more speculative places. At the end of the day I opted for a system that I knew was realistic rather than futuristic - but I still learned a huge amount by doing this.

Thanks

I’d like to thank my Spike McCue, Bobby Genalo, Roopa Vasudevan, Doug Kanter, Craig Protzel and Lia Martinez for their assistance and advice during the project. I’d also like to thank Steven Dean for leading the DIY Health class with insight, energy and enthusiasm. It was a lot of fun.

MetroChange

For more information about MetroChange see metrochange.org. There is a lot of good documentation and background information on the MetroChange blog. MetroChange is a collaboration with Stepan Boltalin and Genevieve Hoffman.

Pilot

Pilot is a speculative system at the moment; I have been told by many, many swimmers that this is a device that has real potential. I have a clear sense of what it would take to make this into a real device, something I’m going to attempt to make happen in the next 6 months. I will keep you posted.

Forever

Designed as a critique of The Spectacle with the desire to avoid using the media of spectacle, Forever offers an isolating, immersive, indescribable experience for one person. The project is a collaboration with the amazing Lily Szajnberg and came out of the Recurring Concepts in Art class at ITP (one of the best classes I have ever taken). 

Winter Show

I’ll be writing more about these three projects in the days to come, but I’m proud of all of them. Please come and see me at the ITP Winter Show. Find me or tweet @paulmmay. See you there!

In my DIY Health class we have been asked to design a system that allows us to “take stock of ourselves, set goals and act upon our health and lifestyle enabling systematic, ongoing analysis and progression towards goals”. 

The project is meant to be speculative - the end result is a thoroughly designed and described system and a video of how the system would be used. We can imagine ourselves at a point in the near future if we like.

My Concept

My goal is to design a device or system that can help me improve my swimming form by monitoring the position of my body in the water, offering feedback in real-time and facilitating analysis of my swimming form over a longer period of time.

Motivation

I am really interested in the complexity of swimming as a physical activity; it’s pretty startling that we’re able to do it at all. When I swim front crawl I am trying to balance the movement of my hands, arms, body and legs to move through the water. When I’m swimming effectively, these things are in balance and become unconscious actions.

When I need to improve my swimming form, I have to become aware of a particular aspect of my swimming stroke to coax it in a different direction - I receive an instruction before I swim from a coach, I drill a particular aspect of my form, or (on very rare occasions) I look back on a piece of video.

The initial motivation behind this project is the opportunity, as I see it, to provide feedback to a swimmer in real time; connecting them to their physical movements in a natural, intimate way. By doing this, it’s my hope that swimming becomes more efficient (expending less energy as I move through the water) and more natural (improving gradually while I’m in the water in a way that reduces the need to constantly drill individual aspects of my stroke).

In my early research for this project I’ve looked at the ways in which fish and other sea creatures move through water and fluidly make adjustments to their speed and course; I like the idea that in some alternative evolutionary path humans could have retained a much closer connection to water - maybe we would have the capabilities to move through it in more fish-like ways, or maybe we’d be able to communicate or breathe in water. There may be natural systems that I can learn from as I do this project.

I’ve also been inspired by the writings of Jacques Cousteau; In The Silent World he describes his first use of the aqualung in such a beautiful way:

From this day forward we would swim across miles of country no man had known, free and level, with our flesh feeling what the fish scales know

In Cousteau’s case, technology had literally allowed him to enhance his evolved, human capabilities. I have started to think about the goal of my system in a more lyrical sense - the system should make me feel more connected to the environment of the water and allow me to become more fish-like.

Key Components

The key component of the system will probably be a wearable device, worn during swimming, which measures angles of rotation around a number of axes, and motion. The device will give me feedback in the form of vibration and/or sound to signal good form or a correction/action of some kind.

The second component of the system is a communications layer; essentially a way to get information from the wearable device to a computer. This could be done using a physical connection or a wireless link.

The third component of the system is a piece of software that allows me to configure the wearable device; setting new goals/parameters and analyze my previous swimming sessions. This could be a standalone piece of software, or it could be a web service. It might have a social component or it might not.

Next Steps

I want to design my system to monitor a small number of key physical components of the swim stroke so I am doing research into the biomechanics of swimming in both humans and animals.

I’m also mapping out how the system would work from end to end - from the moment I get up in the morning to go to the pool to the analysis after my swim, building a picture of my swimming form over months and years. It’s an exciting challenge.

I’ll write about the project as it progresses.

Apparently, on the day that Ireland agreed to a rescue package from the IMF/EU (21st November 2010) I tweeted 41 times - the most tweets in a single day. I remembered sitting here in New York watching the press conference online. I was tweeting and responding to people in Ireland. I was pretty upset and disappointed - things at home just seemed to be falling apart.

In a previous project, I saw that the most coverage Ireland had ever received in the New York Times was in the same month, November 2010. The coverage of Ireland’s bailout in the New York Times dwarfed coverage of the peace process, tourism, sports, day to day politics.

I had no idea that two peaks from these two unrelated projects would align. I was reminded how useful it is to have some understanding of how to work with data. I get a sense that these abilities will be even more relevant in the future as we try to draw together a sense of our lives and our histories using data; we’ll need to know how to draw on memory, privately held data, personal public data (like our tweets) and news sources to add context and perspective - drawing it all together to gain insight.  I’ve said this before, but today’s coincidence just made it feel much more real.

The Missing Millions in the MetroCard System

The MetroCard system has evolved over nearly two decades - it uses some pretty old technology (a magnetic stripe on the card is read at a turnstile, which updates the value on your card) and a slightly odd fare structure ($2.25 per ride plus a 7% bonus when you spend $10 or more on a MetroCard) which combine to create some unfortunate results; small amounts of value left on cards when they’re discarded.

According to the MTA the small amounts of value on cards when they’re thrown away or lost adds up to $52m (yes, million) per year. So much money goes missing that the MTA are set to introduce a $1 green fee to encourage riders to hang on to their cards. Our project’s goal is to examine the causes of this “missing” money in the MetroCard system from a number of different angles. After all, $52 million could be put to better use and $1 per card seems like an awful lot of money at scale.

The Ever-Changing MetroCard Pricing Structure

(This section is a little dense, but bear with us.) The MTA Metrocard was first introduced in 1993 with a trial run of 3000 cards. In 1998 they introduced the MetroCard Bonus giving you a 10% bonus on cards of $15 or more. At this time a single ride using a MetroCard cost $1.50. In May 2003, subway fares increased from $1.50 to $2.00. The bonus increased to 20% for any ticket amount $10 and over.

In March 2008, the MetroCard Bonus decreased to 15% for purchases of $7 or more. In 2009, subway fares increased from $2.00 to $2.25 per ride. In December 2010, the bonus decreased to 7% for tickets of $10 or more. This is the pricing structure that applies to the pay per ride MetroCards today.

The MetroCard system has changed almost constantly since its introduction - and like most transit systems has gotten significantly more expensive over time.

The $10 MetroCard is Not Your Friend

Even a quick scan of the MetroCard pricing spreadsheet we’ve put together shows some significant amounts of remaining value on cards. Let’s look at the popular $10 MetroCard as an example; a card that’s popular among visitors to New York.

A customer purchases a MetroCard for $10 and gets $0.70 of a bonus making the card worth $10.70. At a price of $2.25 per ride, the customer gets 4 rides leaving $1.70 remaining on the card. To get another ride the customer needs to add at least $0.55.

Adding value using the MetroCard vending machine is a significantly longer process than buying another $10 card - and adding whole dollar values like $10 and $20 still gets priority. 

Getting the Most from MetroCard

Our goal is to find ways of getting the maximum number of rides on the New York Subway while reducing the small amounts of remaining value left on our MetroCards - avoiding problems like the ones the $10 card introduces.

Working in increments of $1 and $0.25 we identified the MetroCard values that result in the maximum number of rides for consumers with as little remaining value as possible. We also identified the MetroCards where the value of the remaining value is covered by the MTA’s own bonus.

We think the results are interesting. Check out the breakdown of the MetroCard pricing structure in this spreadsheet.

As you’ll see, the $10 MetroCard offers the same number of rides as a $9 MetroCard. There’s no “bonus” but 4 rides is still 4 rides. A $20 MetroCard gives the consumer a bonus of $1.40, 9 rides with $1.15 remaining on the card. They could spend $19, get 9 rides with only 8c remaining on the card.

There are several price points where the user gets the same number of rides as they would at a higher price point, with minimal remainder - a consumer could spend $14.75, get $1.03 as a bonus - 7 rides with 3 cents remaining; any remainder is essentially covered by the “bonus” given by MTA. You pay less, you get the same and if you happen to lose or throw away your card the MTA are paying for the remaining value on the card.

Implications

We start to see that some of the problems of missing money in the system are a product of decisions by MTA themselves. The pricing structure leads to uneven values and remaining value on cards. The design of the vending machines works contrary to adding small values, or getting a specific number of rides. It’s not wrong, in our opinion, to start questioning whether adding another $1 on to the purchase of each MetroCard is valid.

Next Time You’re In New York

Use our MetroCard pricing spreadsheet to find a card that offers you enough rides for the time you’re in town, and reduces the chances of you being left with money remaining on your card.

Next Steps

We’re continuing this project - we hope to find more ways of eating into these odd problems with the MetroCard system upstream. After that we’ll devise ways of using any remaining value that is left on cards in a more productive way.

This is a semi-academic examination of an exhibition of work by Manfred Mohr. For decades Mohr has used computation in his artwork; he started at a time when computers were large, rare and exclusively logical beasts - the idea of using them to generate art was incredibly novel. The critical responses to Mohr’s use of computers in the production of his art ranged from delight to outrage. This exhibition brings together a collection of his works that explore the intersection of computing, language and aesthetic.

My response is organised under three concepts: The Artist Steps Away from the Canvas and Observes, The Artist Reduces Complex Phenomena to Simple Elements and The Artist Collaborates With Non-Human Actors. I refer to some other works we were looking at in class at around the same time, including Robert Rauschenberg’s Open Score; one of the most famous pieces of performances art ever staged which included groundbreaking technology for its time. I also refer to work of Constructivist artist and architect Laszlo Moholy-Nagy whose work involves a studied investigation some of the essential elements of art; colour, light, line, movement and shape.

The Artist Steps Away from the Canvas and Observes

For me, Mohr’s process and work are an example of the movement away from the artist as the sole, literal producer of the artifact (holding the paintbrush or wielding the chisel) towards a model of creation where the artist is the originator (providing an opportunity or a vision), the referee (the designer of rules and constraints), the over-seer (providing ongoing impetus, clarification), and then the observer (watching the elements of the work come together - along with the audience).

Mohr establishes a project, and like an engineer - sets about designing the limits/traits of his system. This image shows the logical flow for the computer code that will give instructions to the computer and plotter drawing one of his pieces.

This proto-code would then be translated into working instructions for the computer. From this point, Mohr’s involvement in the production of the final artwork is that of a collaborator with the drawing machine - he chooses the paper, mounts the pen in the machine, and makes the decision to proceed. 

For me, this photo captures the idea of changing role of the artist. He is smoking a cigarette while the arm of the plotter moves over the surface of the paper, drawing the image. Mohr is keenly interested, watching the image emerge as an audience member might - he is deeply invested in the work, but his direct involvement in its making has already happened.

What I find exciting about this image is that it captures the feeling of something unfolding that can’t be stopped. It also captures the sense of anticipation and chance. With any computer program there is the potential for the unexpected - times when something is lost in translation between intent and outcome. In the photo, I get a sense that Mohr is watching to see what might happen, as well as what he knows will happen. Mohr speaks about the influence of chance in his artist’s statement:

Even though my work process is rational and systematic, its results can be unpredictable. Like a journey, only the starting point and a hypothetical destination is known. What happens during the journey is often unexpected and surprising.

For me, there’s a direct parallel between Mohr’s algorithmically drawn works and Open Score by Robert Raushenberg. Rauschenberg puts himself in a similar position to Mohr; he is the driving vision behind the piece, he designs the constraints/rules that will apply - those of a recognisable tennis match, but the outcome of the piece is the confluence of preparation, the work of collaborators (human and technological) and an element of chance.

Like Mohr’s work, Open Score expresses the potential for something with a defined set of rules and a repeating structure to offer something unexpected and unrepeatable.

The key difference that I see between what Mohr and Rauschenberg are doing is in their attitude to the audience. In Open Score, Rauschenberg amplifies the experience of the tennis match using sound and light - he gives them a hyper-real tennis match that involves their senses as no other game of tennis ever has. There is a clear sense of the impossibility of this work being repeated and a question as to whether its possible to separate a spectacle from the reactions of the crowd.

In Mohr’s work, while an audience may be present to see the pieced being produced by the computer and plotter - they can only watch as the process unfolds. Their may be captured elsewhere (such as the poster from the 1971 show) but this emotion doesn’t find its way into the “performance” underway (interactivity beyond simple choices/interventions [e.g. an audience member unplugging the drawing machine]).

The Artist Reduces Complex Phenomena to Simple Elements

The second concept that I feel is in evidence in Mohr’s work is that of a reduction of complexity/form to simplicity/geometry.

Mohr makes the direct conceptual link between the reduction of shapes to their individual lines with the reduction of words to their individual letters (Fig 3).

This conceptual link is mirrored in the process used to make the work. By using a computer and plotter to draw his images, Mohr puts himself in the position of literally needing to translate shapes into their most basic elements. Move the pen left two inches. Lift the pen. Move the pen up one inch. Put the pen down. By breaking words to their letters, and shapes to their lines - Mohr demonstrates that complex forms are the products of basic elements, repetition and human ingenuity .

There are clear parallels between Mohr’s attempt to sub-divide forms into their most basic elements and Moholy-Nagy’s effort to categorize and classify the essential elements of colour and material. I feel as though Moholy-Nagy’s goal was a purer, more potent form of expression stemming from a return to the atomic level. In a sense, Moholy-Nagy’s work is about the future, but calls out to the past. Go backwards to the origin to know how to proceed.

In Mohr’s case I don’t get a sense that he is saying the work is more powerful because it is formed from very simple instructions - I think he is making a statement that the language of the machine (code), producing works from a simple visual alphabet (geometry), has limitless potential for complexity and depth; a call to consider a collaboration with the non-human in the production of something inherently human - language, and a call to judge this new language on the same terms as we judge existing language.

The Artist Collaborates With Non-Human Actors

The most literal concept that underpins Mohrs’ work is that of a collaboration between man and machine in the production of art. It’s unsurprising that in his 1971 show, the feedback from the audience was mostly negative - with the dominant sentiment being that that art was a human endeavour, and not one that could simply be generated by a machine.

It’s hard to look at this reaction from the current perspective - simply saying that the audience was wrong and Mohr was right, given the centrality of the computer of many cultural and artistic works we see around us today.

A fairer way to look at the reaction he got at the time is maybe to look at any of Mohr’s artistic predecessors and examine their use of what would have been considered “state of the art” technology. An example I like is Van Gogh’s examination of his process and his tools. The image below shows a piece of one of Van Gogh’s many letters to his brother Theo. In this letter he discusses the merits of different type of Faber pencils.

Mohr takes this concept further; his pencil has capabilities beyond those of Van Gogh’s and the existence of these capabilities poses questions about the location of the “value” of art - is it in the thought, the process, the artist? Is it solely located in the end product? Is it possible to separate the art work and the tools used to make it?

Like Rauschenberg and the diverse group of artists and scientists involved in EAT, Mohr’s work could not exist in its end format without the use of technology. The key difference between Mohr and Rauschenberg/EAT for me is maybe the attitude to the technology itself - I feel as though with EAT the technology was developed in service of the idea and the desired outcome (a sense of direct engagement and magic) - but that the technology was fluid/interchangeable. I don’t really get a sense that technology itself is under scrutiny.

With Mohr I feel as though he is making a considered approach to work within the limits of available technology (rather than finding any technology that was capable of offering the end experience/artifact). He also seems to be making a direct comment on the huge potential of the technology itself. Mohr refers back to a specific technology - like logic, repetition, the computer, the pen, the letter or word.

Final Thoughts

Mohr’s work really surprised me - it felt like a poetic collaboration between man and computer made at a time when any type of digital computation was rare, in any sphere - let alone in the production of art. 

The concepts I see in his work; the artist as orchestrator but not direct “maker”, the reduction of forms to simple elements and the use of technology are strongly in evidence in contemporary art and culture; he was clearly ahead of his time.

A map of your body, delivered each morning to your door. The parts of your body that are probably okay today are shown on the map. Anything that might be amiss is also shown; as a prompt for you to explore further. Today you might want to explore what’s going on with your right shoulder.

Message Tattoo

Each day a tattoo emerges on the skin of your body. The tattoo gives you a friendly message about all the things that are currently not understood about your body. Today, your body is letting you know that there’s no real consensus on whether, when or how you should warm up for exercise.

IRC With Your Body

The first thing I do in the morning is get on my computer to chat with the different parts of my body. Today, my shoulder isn’t feeling great again. Something is also up with Pancreas.

Futility is Freeing

Let death find us as we are building up our matchstick protests against its waves. Alain de Botton - The Pleasures and Sorrows of Work

I chose Alain de Botton’s video because I enjoy his brand of accessible contrarianism. In his Sunday Sermon, and in his books, de Botton draws attention to the futility of existence and the absurdity of our day to day lives.

To me, his pessimism feels both human and invigorating - he pays tribute to the human spirit and to all that we’ve achieved in the face of probable tragedy and certain death. His goal, if he has one is to motivate us to focus on the things that matter while we have time.

Just Try Your Best

Some of the things de Botton talks about are directly applicable to the design of self-care systems. Obviously, his work reminds me that however well we design systems to help people maintain or improve their health we are simply forestalling certain death. Death is, I suppose, the ultimate design constraint. For me, this prompts me to think about designing systems we should do so with a sense of humility in the face of certain failure; aiming for small amounts of progress rather than complete mastery, lowering expectations - or at least keeping them in check, prioritizing relationships and leaving room for warmth and pleasure. It should be possible to design a system where the user can fail, but still prosper and learn.

We Rolled a 6

Another really interesting theme in de Botton’s work is the influence of chance/fortune in our lives. We simply don’t think about our successes as being accidents, and we look upon those who have failed as - failures - rather than people who simply rolled a 1 instead of a 6. Now, when I think about designing a self-care system I think that being conscious of randomness, chance, serendipity and tragedy will be very useful. Instead of thinking “the user will do” I could instead thing “the user might do” and “the user probably won’t”.

De Botton’s perspective on life offers a very strange starting point for us - we start out on a design challenge knowing that success will have a lot to do with luck and that ultimately we’ll fail. I suppose the beauty of life, and the absurdity of life, is in the trying. So, let’s try.

I spent just over a month building my own data service; providing statistical and governmental data from Ireland in a format that would be useful for my project and anybody else who was thinking of the same thing. I wrote some simple applications using open source software that people could download and modify - creating their own applications. I proved that the skills and effort involved in doing this should be well within the reach of a government department, state agency or company - the lack of care and attention was completely voluntary.

In the last year there has been some progress; some academic bodies in Ireland have started to highlight the importance of open data, some other voluntary projects have started, and there have been some small steps made by government agencies to make data available in better formats.

Outside of these modest steps, none of the problems I wrote a year ago have been solved. Ireland is just not anywhere on the map when it comes to offering its citizens current, accessible, coherent insight into the workings of government or the financial system.

My own little attempt to make progress is coming to an end because the effort involved in maintaining the data is pretty enormous, and other projects need to take priority for now. I learned a huge amount creating this project and I hope that truly open government in Ireland becomes a reality sooner rather than later.

Observation

As with my other observations, I took up two positions (1 and 2 on this little map) - just so that I could get a sense of the space from different perspectives.

I sat at the corner of 17th and Broadway (1 on the map) observing the busy intersection; a confluence of people coming from and going to Union Square, and people walking along 17th avenue. Into this mix you have an on-street seating area, cycle lanes, a roadway, payphones, street vendors and a lot of busy shops. It’s a complex little area - oh, and in case I forget; there’s also Rob Pruitt’s statue of Andy Warhol sitting smack bang in the middle of people eating their lunch.

The second location is actually on Union Square, at the seating area in front of the grand entrance to Union Square park. From here I could observe people entering the Northwestern area of the square, making their way North up Broadway, or into the open plaza. This is a far less dense space than the first observation point - people enter and exit the space slowly, so it’s a chance to see where and how they walk through the space.

Key Observations

My key observations from the first location relate mostly to how people use the space. It’s a space of transience, but also density. People walk through the space in orderly bubbles/tides - which are broken up by the rhythm of the traffic lights.

I also noticed that tourists and (apparent) residents use the space differently. The tourists are drawn along the orderly lines of the pedestrian crossings, many of them walk directly towards the statue of Andy Warhol. The residents exit the pedestrian crossings early, breaking off in a particular direction - with some definite end point in mind. I just thought it was interesting to see how you could discern different profiles of movement from carefree visitors and those with a few minutes to spare from work.

The second observation point told a completely different story. From here I could see how people entering the large open square essentially all follow a small number of paths. They enter and exit far more continuously than at 17th and Broadway; it’s less clear that they are being controlled by the rhythm of a traffic light, or even the arrival of a subway train. I noticed that between the small number of desire-lines/paths there’s essentially a dead zone where few people walk. Taking this path would lead them to no place in particular in either direction - so it sees less footfall. This was interesting because, if our intervention in the space sought to break up a flow of people it’s pretty clear that there are good places and bad places to do this. On the other hand, if we want to stay out of the way we now know where to stand/wait.

So, they are some quick notes from the morning. There are more photos and notes on Flickr.

Summary

In this talk, Leadbeater encourages us to think about things that we’re just not encouraged to think about, or speak about (or let enter our heads in any way). Let’s face it, for most of our lives, our later years barely get a thought, and death is just off limits.

Leadbeater encourages us to grapple with the certainty of ageing and death, and to fundamentally re-think our later years. For him we should re-imagine the ageing and dying process founded on two principles; participation and relationships.

Towards the end of our life we should be able to participate and contribute. We should be able to do the things that we love doing; our lives should be about capabilities, not about infirmities. We should be able to make active decisions about the type of death that we want and don’t want. We should grow old while maintaining strong relationships with others, conversing with others - rejecting the idea that the highest goal we should strive for by the end of our lives is efficiency and professional success.

My Reaction

I love this talk because Leadbeater talks about an incredibly sensitive topic in a completely open and humane way; It is hard enough to talk about death, let alone to talk about death in front of an audience - using the death of your parents as the key reference.

The talk really resonated me. I’ve had some experience of the death of a family member, and I suppose the way I think of ageing and dying is fundamentally shaped by this experience. For me, the process of ageing and dying haven’t really been positive things to think about; they go hand in hand with medical intervention and the loss of capability. His talk has shaken that perspective a little bit, and encouraged me to actively consider the type of life that I want.

In a broader sense, the talk has also made me think about the things that I design. At ITP this semester I’m working to re-think the way we participate in urban environments and to design health devices and services. The talk has just encouraged me to think about these two challenges from alternative perspectives. Maybe routes that that feel uncomfortable feel that way because they are more necessary and potentially more productive. I need to step beyond what I’m familiar with, otherwise all I will ever design will be some version of my own perspective on the world - no matter how much research I do.

The fact that while doing research I was learning how to do research never really bothered me - that’s life. What did bother me was the sense that sometimes I wasn’t being honest or fair in my research - that I was excluding uncomfortable options, or didn’t understand how to deal with people fairly - basically that I didn’t have a handle on the ethics of design research.

Now I see more and more research being done to inform design projects, and I’m sure other people have experienced similar issues along the way. This weekend, after completing a little observation study, I jotted down a few thoughts; five ethics that I think now underpin how I do design research.

1. Necessary

When I do design research, it has to actually be necessary. It needs to hurt that I don’t know things that I need to know. Design research is the way that I find answers that are material to the success of the project. It has to be about more than just getting to travel, getting to know people, building relationships or showing that I’m smart.

2. Designed

When I do design research, it itself has to be designed - I need to have crisp, clear questions that I’m trying to answer. I need to choose the best way to answer those questions, using just the time and resources that I need. I need to have a clear approach to assess my research against how well those questions are answered. There is nothing to preclude serendipity, or the identification of other worthwhile questions, or all the great stuff that happens when doing research - but those aren’t reasons to start out without clarity.

3. Appropriate

When I do design research I need to use the right tools for the right job. For every question, there is probably a right (or at least better) way of answering it. I need to become comfortable with the design and implementation of experiments. I need to be comfortable with careful reading. I need to understand data. I need to be comfortable observing, interacting with and interviewing people. Even as a human-centred designer, I need to acknowledge that most research looks more like reading a book than ethnography.

4. Fair

When I do research I need to be open to having my own preconceptions challenged; in fact I should worry if I get to the end of a research exercise that doesn’t challenge or progress my initial thinking. As well as this basic commitment to fairness. I need to treat people I encounter with integrity. If they are helping me, answering questions - they need to understand the context of that assistance (what I’m doing, why, for whom) and they need to be able to revoke that participation no matter how inconvenient it might be for me.

5. Acted Upon

If my design research is appropriate and planned properly then I should be able to act upon its findings without fear. I may not know everything, but I didn’t set out to answer everything - so that’s okay. I should be able to say that because I know X from my research I did Y in my project.

That’s It

I can’t say these points cover everything, but they are the corollary of most of the things I’ve done wrong over the years. I also can’t say that I hit the mark on these all the time, but I at least have a grasp of what I should be doing and why. Maybe you’ve struggled with your work too, and maybe you have a different take on ethical design research. Let me know - whatever that case - I’d love to hear your thoughts.

My focus in this observation is to document the spaces where people congregate at the intersection and their interactions (at a very broad level).

I broke the area I was observing into 6 sections (shown in pink on this rough map). I chose two observation points; a bus shelter at 1-2, and a table outside a cafe at 5-6.

I made observations from these two points; sitting at point 1 for 60 minutes from 7.30am to 8.30am, then at point two for 30 minutes from 9am to 9.30am. I also spent 30 minutes taking photographs around the area (from 8.30am to 9.00am).

The view from observation point 1 on 7th Avenue.

The view from observation point 2 on Flatbush Avenue.

I decided to code the behaviour I saw in very loose categories:

• Stop - a person stops and lingers for some reason (e.g. buying something from a street vendor)
• Wait - a person stops and waits (e.g. waiting for a bus)
• Look - a person examines an object on the street (e.g. looking at a bus schedule)
• Coup - a pair/couple interact with each other. (e.g. a husband and wife interact at the bus stop)
• Group - a group of more than two people congregate (e.g. a group of friends interact with each other)
• Brief - a brief interaction between people who pass on the street (e.g. two people pass and say hi to each other)

Again, these are loose categories with blurred edges. There are some cases where an observation could fall into more than one category. I’ve tried to get as close as possible to the interaction I observed - from my subjective point of view.

I decided to use a simple ethogram to document this activity. For each observation I recorded the time, the section number (from 1-6), the category of behaviour and some notes. Here’s a snippet from my many, many pages of notes:

Data Recorded

I made 81 observations in the 90 minute period - which I am now transcribing into a spreadsheet. I took 15 photos, which I’ve added to the set of photos taken in my first observation.

Key Observations

As the morning proceeded I noted down some key things I was seeing:

• Most people are in transit, moving through the area - remaining for a few minutes (2-5 minutes).
• The key period of activity lasted from 7.30am to 8.45am, after that it became noticeably quieter.
• The bus shelter (area 2) is a hub of activity where people congretate.
• I observed lots of family and social interaction at the bus shelter (area 2).
• The fruit seller on the corner of 7th and Park Place is placed at a very good location (area 3).
• As people walk Northeast up 7th Avenue, they turn onto Park Place (area 3) heading towards the subway station on Flatbush (area 4), passing the fruit seller (area 3).
• The bus service that stopes on Flatbush (area 5) is far less frequent or popular than the service stopping on 7th Avenue (area 2).
• People sometimes use street furniture/objects for novel purposes; hanging clothing, resting bags.
• The green space in the middle of the intersection - a small stand of trees - isn’t traversed by many people. It’s essentially a traffic island.
• The subway stop (7th Avenue BQ) is like a magnet, drawing people up the block - chiefly from the direction of 7th and Park Place.
• People interact very little with objects that don’t relate to their commute. There is very little interaction with any other object on the street.
• Interaction with the network layer of the city was almost entirely done through cell phones. One payphone call was observed.

Next Steps

I’m compiling the observations into a spreadsheet so that they can be sorted and counted easily. I’ll then add all the data onto a map, combining the observation of human activity with my survey of networked devices. As I continue to make observations at the intersection I want this map to build up into a useful document of the different layers of the urban experience. 

So, I’m very happy with my morning’s work.

For this series of observations I’ve chosen the block at the intersection of Flatbush Avenue and 7th Avenue (my Northern boundary), South to Flatbush avenue and Park Place - a diamond of three intersecting streets.

I chose this area (which you can see on this map) because it’s a busy commercial hub at the center of a densely populated area. The 7th Avenue subway stop sits underneath the intersection, drawing a rush of commuters in the morning and evening times. Many of the area’s local buses pass down Flatbush Avenue. While it’s a busy intersection, there is still some green space; where Park Place and Flatbush Avenue diverge there’s a small area of trees and plants.

So, it’s a really complex little space with lots of different layers of experience to pull apart and analyse. I took a lot of photographs while doing the survey, which are available over on Flickr and also on a (rough) Yahoo map (to be replaced by something in Arcgis or Google Earth later in the week).

Survey Location - North to Southeast

I started at the Northern boundary of the survey location, and walked Southeast down Flatbush Avenue to Park Place.

On this side of the block there were few (obvious) networked devices at street level (I cover the traffic system separately); some residential intercom systems, a satellite dish and some antennae.

Intercom system - there was one of these on practically every door on the east side of the survey area from 7th and Flatbush to Park Place and Flatbush.

A satellite dish mounted to the side of one building.

One of two roof-mounted antennae on Flatbush Avenue.

A mobile phone mast mounted on the roof of a building at the corner of Flatbush and Park Place.

I noticed some small devices labelled DEP attached to each building on this side of the street, but I wasn’t sure what they were - I photographed them anyway. They are electricity monitoring devices manufactured by the company Aclara, and installed by the Department of Electricity and Power. I want to read up on these some more, but the patent documents seem to indicate that they send information about power consumption back to a central computer; networked meters.

I observed a row of parking meters that didn’t appear to be using a solar panel; I assume they are drawing power from the grid, or using such small amounts of power that they are powered from a battery of some kind. I’m not sure if they communicate information to or from a network. They do qualify as objects that communicate information back to a person on the street.

At the corner of Flatbush and Park Place I observed an emergency telephone box; somebody had put a lost glove on top of the box.

At the same corner pedestrians request to cross by pushing this anonymous button; the plate which (I assume) once had instructions on it has long-since faded.

Southeast to Northwest

Crossing the street, I walked Northwest up Park Place. On top of one building I observed a large satellite dish.

On an adjacent building I observed a mobile phone/microwave transmitter.

I walked down the steps of the Flatbush Ave/Park Place entrance to the 7th Avenue subway stop. The station features the standard MTA ticket vending machine and a smaller, card-only machine.

A little way up the block I see my first conventional telephone system; a dual payphone - which looks a bit worse for wear.

Traffic System

The traffic in the area is quite heavy - Flatbush Avenue is one of the main routes through Brooklyn. As such, there are a lot of traffic control devices.

There are lights at each junction of the intersection; three sets of lights on both sides of the street.

I noticed something I hadn’t seen before; each light had a small antenna mounted to its post.

There were three large traffic control boxes - each one had a similar antenna mounted on top of a vertical post. I can only assume that the traffic control boxes are communicating wirelessly with each other and with the traffic lights, co-ordinating their actions - as well as connecting back to a central traffic control system. 

At pedestrian crossings there are signals communicating to pedestrians when it’s safe to cross.

Other Observations

I was really surprised that I saw no working cameras anywhere in the survey area. One bar did have an infrared camera mounted above its door, but it was clearly broken and was pointed at an angle where it couldn’t see the street.

Similarly, there were practically no display devices anywhere in the survey area - just the traffic signals, some small LED displays on parking meters, and the displays integrated into the MTA ticket vending machine in the subway station.

I also observed that each pillar and post has a unique identifier, usually accompanied by a barcode; presumably this information is recorded somwhere along with information about the location of the pillar/post - and is then used by maintenance crews. It’s interesting to think that there is (probably) a database somwhere noting every single piece of street furniture/infrastructure in some way.

One sheltered bus stop also featured a unique identifier; though this time customers were encouraged to use the identifier when making issues known to the city. 

Final Thoughts

The exercise of cataloging physical items in a space was revealing; I had never really paid any close attention to the items I saw today. So much of the city just washes over me; slowing down my pace and spending time just to observe was really interesting.

I was surprised that many of the networked objects to be found in the survey area were practically invisible (traffic control network, power grid network, roof-mounted antennae) - and that they communicate through (presumably) private networks built for a single purpose.

Next Steps

This week I’ll be visiting the survey area again observing human activity. I have begun to transfer my observations from today into a KML or Arcgis file, which I’ll make available here.

I was interested in the idea of self-monitoring devices that could carry out what today would be considered quite advanced laboratory analysis in a persistent way - carrying out ongoing genetic analysis of our biome or monitoring for specific antigens.

I was inspired microfluidic lab-on-chip systems, genotyping chips, rapid diagnostic kits and camera pills. I was really heartened that the cost these technologies has fallen considerably in recent times.

I took a leap to imagine a time when these devices are so cheap and small that they could persist in our bodies in some way. In all cases, the devices carry out their work inside the body and send data back to the user’s watch, phone or tablet - where the data can be displayed, analysed, shared and acted upon.

Semi-Persistant Genotyping Pill

This pill is essentially a miniature genetics laboratory. The person takes the pill, which travels through the gastrointestinal tract, building a genetic fingerprint of the organisms it encounters.

In the diagram I show a bacterium being captured by the pill, having its DNA separated and analyzed, before being genotyped and data sent back to the user.

Persistent Genotyping Lab/Patch

This miniature genetics testing lab would carry out a similar function to the genotyping pill, but in the bloodstream.

The lab would be inserted under the skin, or during surgery - and would fixitself to the wall of a vein or artery, capturing microorganisms for genetic analysis. This would allow the user to get near real-time information about the strains of viruses they are exposed to and the types of bacterial infections they have travelling in their bloodstream. 

Antigen Lab/Patch

This is a lab-on-chip device that sniffs for particular antigens in the bloodstream. The lab carries specific antibodies that bond with corresponding antigens that arrive via the bloodstream. When an antigen is detected, that information is sent back to the user.

Conclusions

I enjoyed this exercice; it’s nice to just sit and sketch ideas - completely unencumbered by constraints (or knowledge!) - to see what happens. Though these devices are fantastic, I do like the idea that once-in-a-lifetime (or very infrequent) medical testing could become something much more immediate and persuasive. What that means of course is to be discussed and explored.

The Pattern in the Noise

The Pattern in the Noise: Introduction to Statistics and Data Analysis with R is a seven week class taught by Jake Porway. The goal of the class is to teach us some fundamental skills related to the analysis of large data sets using the stats package R. I wanted to take this class because it’ll help me accomplish more with my work in data visualisation - I just want to be a little more confident in my ability to identify and draw out meaning from data sets, while treating source data properly/ethically (i.e. not pointing to things unless they are significant beyond reasonable doubt, or connecting things that aren’t connected).

The next steps in this class are to become more comfortable with R by manipulating sample sets of data, before moving on to look at our own data.

DIY Health

In DIY Health, taught be Steven Dean, we are designing tools to help people gather information about the current state of their health, set goals, encourage action and monitor progress. Instead of working to improve existing health infrastructure (better hospitals, better medical devices), we’re trying to intervene in people’s day to day lives, using smart devices to encourage healthy lifestyles. The eventual outcome of this project will be speculative designs or prototypes of new services/devices that we believe will have positive effects for people.

The next steps in this class are look closely at behaviour and motivation as they relate to health/lifestyle, before looking at ways to design positive feedback loops that help to encourage healthy lifestyles. I will probably try to integrate some of my experience training for triathlon into this class as well.

Recurring Concepts in Art

This class takes me out of my comfort zone! In this class, which is taught by Georgia Krantz, I’ll be looking at the history of major movements in modern and contemporary art. Each week we read and investigate a theme; like the use of technology, reproduction/copying, the body - using significant artists as examples and applying what we learn to new media and design.

I already love this class because it really makes me think about what “new” really means in relation to ITP - where we try, but often fail, to advance the state of the art that looks to merge design, people and technology. The next steps for the class are to continue looking at more themes that occur repeatedly over time - making field trips to places in New York where we can see related works.

Urban Experience in the Networked Age

In this class, taught by Adam Greenfield, we are examining the development of the modern city as a “stack” of people, infrastructure, services and data. We are basically trying to pick apart the modern urban experience, proposing new ways in people can work with non-human actors (connected objects and sensors) and services to live healthier and happier urban lives.

The next steps for this class are to make field trips out into the city observing the environment, people, infrastructure, devices and systems - documenting these different layers of the urban experience using photography and video, before proposing a project in which we’ll intervene at this location in some way - maybe with a service or device. This class is going to be a huge challenge, but it’s really exciting.

Poweful Ideas: Useful Tools to Understand the World

This class is taught by Nancy Hechinger with guest lecturer Alan Kay. Alan is one of the world’s foremost experts on computer science; he co-created the concept of object oriented programming, was lead scientist at Atari, a researcher at Xerox PARC and has done pioneering work in the design of computing devices for children. In this class we are using the concept of “powerful ideas” (ideas that, when arrived at/agreed upon, fundamentally change the way we understand the world - like the scientific method) to propose more human ways to teach and learn.

This class feels a bit like it’s circling the airport, but in the next few weeks I’m confident we’ll land the plane - if you know what I mean.

That’s All

I want to write more about what we’re up to this year, but I am short of time - I’ll be posting updates on classes and projects as the semester continues.

I left you on the arm of my chair in the cinema, and by the time I was a few steps away you were gone. The man checked in the office, he checked in the box. He took my name and wrote it down carefully. I called the next day. Any sign? No.

I was careless and then you were gone; that’s six months ago now.

At first it was weird - being without you. No Radiolab on the train. No e-mail on the go. No tweets. No check-ins. No to-do list. I visit again in the land of the phone call and the text message, the post card and the passenger pigeon.

But over time I’ve learned to cope without you.

I started to read again, yes read, on the train. Real books, paper. Those books I bought, I have now read most of them. The tweets I send and then regret later because, really, I need to lighten the hell up - I send at least 50% less of those than before. Those things I have to do, well I still don’t do the same amount of them.  Now I save seconds by not needing to check the ones I actually do off a list. When I am gone from work, from school, from there and them - I am gone.

You were great iPhone, the freebased nicotine peak of 25 years of computer usage. I can’t say it wasn’t fun.

I don’t miss you. There. I said it.

Background and Objectives

This year a lot of my work at ITP has involved the use of large data sets to tell or discover personal stories. We Are Working On It is a continuation of this theme. Unfortunately my family, like many families, has experienced the devastating affects of cancer across multiple generations. I wanted to use available data and accessible tools to investigate how the amount and type of cancer research has changed during my lifetime. I also wanted to look behind the data and meet with cancer research scientists, talk about their work and understand who they are and what they do.

My goal with this project was to combine both sources - data and audio recordings of conversations with researchers - into one coherent timeline.

Retrieving 1.9 Million Scientific Journal Article Abstracts

The first part of the project involved identifying a source of scientific data, then writing an application to retrieve, store, parse and display this data in a timeline. I decided to use the Pubmed scientific journal database provided by the National Center for Biotechnology Information (NCBI) as my primary source of data because of the range and depth of available data, and the fact that this data is accessible through a very solid API.

My Processing application first passed a search term (“cancer” and related keywords) into the NCBI’s search utility and retrieved scientific journal abstracts as XML - the amount of data was pretty immense (it took over a day to complete the full search).

One of the key challenges was to then interpret and store all this data - as you might expect the hygiene of the data varies the further back in time we look. My application had to construct useful objects (journals, articles, research centres, scientists etc.) from the data and then store these objects in a local database (MongoDB) - where it would be much quicker to run follow-up queries.

Building a Timeline of Cancer Research

The next step of the process involved the presentation of the data in a form of timeline. I spent a lot of time thinking about how the aesthetic of the representation of the data could be sensitive of the subject matter, informative and elegant - just enough design to tell a story without getting in the way. I knew from early in the process that it would be incredibly difficult to take all this scientific information and tell an accessible story - I decided to start with a smaller story about the number of scientists working on cancer research and the volume of work they produce. This seemed to fit well with the parallel track of the project where I was meeting with scientists.

I developed a graphic design that shows the number scientists and research centres were publishing research related to cancer research in a number of different journals. I wanted this quantitative data to read almost like a sentence from the top of the graphic down to the bottom.

I also wanted to take a number of article titles from each year and use them to give a flavour of the type of work being done; I used a lexical analysis technique to weight the terms used most frequently in each year, picking articles that scored highly on this representative scale. A better way of doing this would have been to use the most cited articles from each year, but this wasn’t data I had access to.

I have spoken about my belief that all data representation that purports to represent “the truth” needs to address the source and quality of its data. In each graphic generated by my application I make a statement about any data that didn’t meet quality checks (an example of a failed check would be where a scientist had a first but no last name, or where an article had no title).

The application generated a number of PDF images (showing data for 1980, 1990, 2000, 2010) which were then brought into Adobe Illustrator for final type setting and small tweaks.

The graphics were printed on archival paper and framed. When the framed graphics hang next to each other they show the change in the data being represented over time - with more time it would be great to print and hang each year, but this would have been very expensive. You can see the final graphics (and the many failures along the way) on Flickr.

Interviews at Memorial Sloan Kettering Cancer Center

I spent two afternoons at MSKCC’s Powell research laboratory. I interviewed two scientists; Laura Eccles and Ciara O’Driscoll - from England and Ireland respectively - about their work.

I prepared for the interviews by doing some basic desk research about MSKCC and the work being done there. I developed a short interview script that focused on the scientists’ backgrounds, how they came to be working in cancer research, their day to day work and their hopes for the future.

Laura and Ciara were incredibly forthcoming about their lives and their work; I came away from the conversations with them feeling immensely grateful for their hard work, but also much more aware of the day to day frustrations involved in research - it is not an easy life by any stretch of the imagination.

I have embedded two short excerpts from the interviews here - this audio was played alongside the printed graphics at the ITP show in May 2011.

  We Are Working On It by paulmmay

What I Learned

This project was technically challenging and personally draining at times - but I learned a huge amount. I feel now as though I would be much better prepared for a project involving the manipulation of a huge amount of data - there are just sensible ways to do this stuff, and it took a lot of trial and error to hit on a good approach.

Aside from the technical challenges, I believe that I answered the questions that I set out to answer at the beginning - even if I only scratched the surface of the potential to elaborate on this answer.

I know now that there have never been more people involved in the search for better treatments and potential cures for cancer - that isn’t something I knew before this project. There are over six times more people working in cancer-related fields as there were in 1980. I also know that this international network of researchers publish more work than ever - and that a considerable amount of this work is available to anybody who wants to look for it through bodies like NCBI.

I got to meet two of the people working directly on cancer research - I learned that they are deeply motivated to do the work that they do, but that they are not superheroes. They go to work, work is frustrating, sometimes they make progress and most of the time they don’t. It is deeply heartening to have had a tiny glimpse into the day to day work of these people - it makes the data really come alive for me because I can start picture all the other people grafting away on this impossible problem around the world.

I am proud of this project.

The talk has stayed with me over the years; I come back to it when I think about new projects. I try to work on projects that are connected to a “why” - a motivation, a desire or a problem, and that have a “so what” - an impact beyond the project, even if it’s only to move a tiny idea forward a little bit in my own head.

Last year at ITP I had the chance to work on things that were both personal to me and connected to a somewhat longer term idea; whether it was the idea that my photos connected me to stories that themselves change as time passes, or that data could tell me about where I’m from - I gravitated towards things that matter to me.

This was good and bad. Because the projects were personal and worthwhile to me I worked hard on them, was proud of the outcome and gained real personal insight. On the other hand personal projects are naturally hard to build a team around. I chose to work on my own more than I would have liked and missed out on some lightness, fun and experimentation that could have come from a more social approach.

In the next year I want to continue to work on things that are personal to me and to work on things that have the capacity to move ideas forward - but I also want to find a balance between these things, teamwork and fun. I don’t think any of these motivations are incompatible - but I think it’ll be positive to have a more complex center of gravity to my projects.

Of course, what exactly all that means is to be determined in year two at ITP.

When I talk about the process of design I mean that it is good to approach concepts or problems with a basic toolkit; understanding how to apply a couple of research activities, knowing how to work on a team, finding the right material or tool to use after many years of experimentation.

In year one of ITP I had the opportunity to expand my toolkit, taking approaches I had used to make things for the web, modifying them to make devices, representations of data, audio pieces, essays and lots of other stuff.  I was able to get back in touch with skills that I had let wither, like programming, sketching, electronics, drawing and creative writing. Having a very loose toolkit expanded my ability to get to grips with what was special about each project - it basically allowed me to cope with what was sometimes an intense workload while still doing some good work.

On the less positive side, I also now know that my way of doing things closed me off from ideas and opportunities that I should have considered or pursued further. In the coming year I need to make sure that I am really going out of my way to question how I work, and going out of my way to consider other viewpoints and approaches to projects.

So, that’s item number one for year two at ITP; recognition that each project is actually two projects; the one that delivers an outcome, and the one that expands my capabilities and ways of thinking.

Game Play

Destinations is a game that rewards players for travelling to far-flung parts of the world. Each country is assigned a number of points based on the number of connecting flights in and out of the country; the less connections there are, the more points a country is worth.

Players start in countries of equal value; and use the Destinations web application to book a flight to another country. Flights leave on the hour and flight durations vary depending on the distance being travelled; the shortest flights will last 30 minutes, the longest flights will last 2 hours. When a player’s flight lands, they can can book another flight. Players may linger in a country if they wish. Players do not have to take a flight every turn.

Players will receive updates via SMS to tell them when their flight takes off or lands at which time they should go to the game board, and move their marker to their current destination. Players can view the current state of the world on a large map on the floor at ITP.

Scoring and Winning

Players travel around the world by plane, and receive points for the countries they have visited - once a player leaves a country, they start to earn points. Each player will continue to earn points every hour for a country they have left until another player arrives, and then leaves that country.

If two players are in the same country, the last to leave receives the points. If both players take the same flight out, they split the points evenly.

Thoughts, Feedback

Our goal was to design a game that offered people with limited time, living in a large city an experience that felt like travelling around the world.

Initially we thought about using elements of people’s day to day commute as the core mechanic (maybe rewarding distance covered, or interactions of some kind) but this just didn’t seem like fun, and we also recognised that the time people spend commuting to and from school/work (which in NY usually means taking the subway) is necessary down time, not really a time to be carrying out observation tasks or changing regular patterns.

Eventually we decided to create a game that would offer the feeling of global travel, but playable wherever players were and with light time commitment. We designed destinations as a kind of big board-game which would use the web and SMS messages to provide access and continuity wherever players happened to be when they wanted to play.

We got great feedback from the players after the game finished;

• The texture of the game, the aesthetic was really enjoyable - players did get a sense of competition, travel, fantasy.
• The game was fun; all but one of the players continued for the full two day game.
• People enjoyed the updates by text messages (“Hello Michael, your flight to Cuba has just departed from Iceland. Pack up those parkas and get out the sandals!”), but in a way being prompted to play removed the need for players to keep track of time themselves
• Having an online map would be good, rather than having to go to the board to see the state of play. This is something we talked about, but didn’t have time to implement.
• It’d be good to think of making a better experience for spectators.
• More players would help fill in the board, increase the sense of pressure - we had 7 players, 2 or 3 more would have been really good (or a smaller area of play).
• A way to interact with the other players - a sense of communication between players during the game - would be good.
• The game appeared to reward constant play - just continuing to book flights and travel each and every turn. The flip side of this is that the game punishes players who don’t play constantly. There is probably a tweak here to make it acceptable for players to miss a turn without falling behind.

Feedback from our instructor, Greg Trefry was also very encouraging:

Great job with Destinations.  This game had a lot going for it: elegant simplicity in the design, a mechanic that really took advantage of persistence and smart technical execution.

The basic mechanic of trying to skip around the world to accumulate points was great.  And then tying your ability to move to length of flights and departure times was great.  It’s odd because the act of going to the airport to catch an actual flight is such an awful tedious, fraught experience.  Yet the game is great because it lets you dream about traveling without the hassle of arriving two hours early and taking off your shoes.  Finding this little bit of content that marries well with your game design can really be the secret sauce in game development.

All in all the feedback was very positive; we are now considering making the game available more widely - with some changes obviously.

Thanks

This was a great way to end our time in the Big Games class; which was by far the most complete, and most intriguing class I took this semester - thanks to Greg for making it such a great class, and to the rest of my classmates who made a lot of fun stuff this semeseter.

Huge thanks to Steve Klise who worked incredibly hard on the web component of the game - he was a real hero on this project.

Introduction

Hello, thanks for this great opportunity to talk to you today - My name is Paul May, and I’m a student at the Interactive Telecommunications Program here at NYU.

My goal with this talk is to continue a conversation that’s happening at ITP right now. That conversation is about the ways in which information/data about our day to day activities is gathered and stored. It’s also about the ways in which personal data and other large data sets - census data, seismic data, political polling data, are mapped into other forms - represented as graphics or sound or sculpture.

We think this conversation is timely. So much of our lives now are spent transacting and interacting with data. If you open a newspaper or website and it’s highly likely you’ll find a slickly produced infographic. Television news has become a mash-up of live footage and data-driven graphics. When we interact with our families on Facebook, we’re interacting with pieces of data about them - data has become a cultural artifact and something we exchange in return for goods and services.

My Projects - Moving Away to Gain Perspective

So, when I came to ITP I think I did what a lot of people, including many Irish artists like Joyce and Beckett, do when they leave their home - they look back on where they’re from with a new sense of detachment. Moving away has been an opportunity for me to think about home in a new way. Some of my work in data representation takes data related to Ireland, and tries to make sense of it, to tell stories with it. In doing this work I learned a huge amount.

My first project, called Irish Data - started out as an attempt to examine the cause’s of Ireland’s recent problems. Ireland, as you may have heard, has been experiencing an acute financial and social crisis. I wanted to examine data relevant to this period of time, to see what I could discover.

I found relevant data incredibly hard to come by, and where data was available it was often in outmoded formats, structured in ways that could almost have designed to make the data difficult to use. The project became much less about representation of data, and much more about trying to grapple with the messiness of data. I took this messy data, cleaned it up, created my own repository and made it available through an API - then finished by making some very basic graphics based on this data. This is just one example of data taking me in directions I hadn’t anticipated.

In another project called From Over Here I wanted to examine how Ireland is perceived from the perspective of my new home here in New York. I suppose like most people I have a thoroughly unscientific idea in my head as to how my place of origin is perceived. I used data pulled from the New York Times to show the amount of coverage of Ireland, and the types/themes of articles. I built this data into a physical form so I could walk around it, manipulate it, find stories. I found that coverage of Ireland is reasonably narrow, focusing on some not-so-positive topics - with our recent financial woes getting more coverage than any topic.

Lessons Learned

In doing these early projects, I learned a huge amount - firstly, as somebody with only basic programming skills, I found that the technical barrier to working with data is incredibly low. Each of us probably has access to a computer capable of accessing, storing and analysing huge quantities of data. Tools like Python, Processing and R - supported by extensive online resources and tutorials - make it easy for us to get started with our own data representation projects.

On the flip side of this reality are the aggregators of data, governments and corporations - who have never had greater access to valuable data about citizens and customers, or greater access to the tools required to manipulate this data. I suppose doing these projects made me acutely aware that we are in a position of incredible weakness when it comes to asking for this data; anything we’re given is essentially afforded to us, and we have practically no insight into the processes by which data is manipulated before it gets to us - we can’t see behind the curtain. We are, I think - too kind to governments and companies who hold data, and either don’t make it available, or make it available in limited/messy ways.

The Value of the Journey

The most important thing I learned in these few projects was that it’s possible to draw a huge amount of personal, resonant insight from large sets of data. It was possible for me to engage with data about Ireland’s financial crisis and for that to be an emotional and informative experience. Before doing this work I wouldn’t have associated emotion with data.

I also realised that, while the end products of the work are aesthetically pleasing - it was the process of engaging with data (finding it, gathering it, sifting through it) that helped me to learn as much as I did.

I think this is an important point; In recent weeks there has been a lot of discussion about the merits of creative representations of data versus more utilitarian ones; slick graphics versus cool charts. While this debate is really important - it feels as though we are arguing over the shape and colour of the tip of the iceberg.

When I look at a photograph in a newspaper, I don’t think that it represents the sum of an entire situation - I understand that it’s an imperfect product taken from one point of view. I’ve learned to question its aesthetic composition, the way it was cropped, and I understand that it captures a moment in time that might have been decades or centuries coming. It doesn’t matter how “good” the photograph is - it’s a product of the information and processes that came before it, representing one possible outcome.

Doing these projects made me realise that data representations are very similar; the underlying story of how that data was gathered, by whom - then how it was treated before it got to me is crucial. The things that are highlighted or omitted are really important, and what I make - my graphic or product - is just one outcome. Unfortunately, we see very little of this noise when we’re presented with infographics online and in print. They are clean and tidy, generated, clockwork - inevitable outcomes. My proposal today is that where we are presented with an infographic that says nothing about the process by which is was constructed, we should treat that graphic as a purely aesthetic product.

Thankfully, we have an opportunity to work through these issues for ourselves, to take up the tools and data behind these products - to tell our own stories and draw our own conclusions, and in doing so we become more informed consumers of infographics and representations of data.

Closing Thoughts

I’m really glad that I took some time to do these simple projects and I think this stuff really matters. Every day we write the most intimate diaries - where we go, what we do, who we meet along the way, what we buy, who we love. We write these diaries in the data stored in devices, communicated over networks - stored by governments and companies. In years to come I think it’ll be very useful for us to have the skills and insight needed to gather this data on our own terms, to re-combine and interpret it; to tell our own stories.

So, maybe after you leave here you might ask yourself a few simple questions: What you are most passionate about or what question most intrigues you? What data exists that is related to this passion or question? How will you get this data, how will you treat or change it, and then finally - how will you represent it in a form that is meaningful to you?

The possibilities are really intriguing, and I am sure you will learn a huge amount - so I leave these as open questions for you, as a starting point of a journey - and most of all I really thank you for your time today.

Get to the viewer. Claim a space and invite people in. Problem; the artist becomes a person in a fixed position. A priest.

I was really interested in the way he talked about his work as a series of attempts - he never seemed to truly happy with his work; every expression was followed by doubt and a new attempt to get closer to whatever felt more honest.

What bothers me about the last thing I did?

When you’re born tripping, you don’t know you’re tripping. The possibility of a more considered engagement with media is worth pursuing.

Doug Rushkoff came to speak to us about his book Program or be Programmed. I don’t agree entirely that absolutely everybody needs to know how to program in order to engage with new media, but I take his point - that a basic understanding of the tools and processes used in the construction of media is very useful.

Rushkoff’s talk made me want to read Amusing Ourselves to Death again - no bad thing.

I’ll admit - I reacted negatively to the phrase Atlas of Homuncular Flexibility. As smart as I think I am, I’m not smart enough to let people finish their talk before putting them in the crazy box.

This is ten minutes later.

Lessons learned; tone down the snark, listen, value people who appear to have arrived from other planets.

Concept

The original concept was to create an augmented reality game based on the structure of a 419 e-mail scam; we liked the idea of using “real life” events, then blending them with a written narrative.

Over several iterations of the story we hit on the idea of using e-mail scam as an introduction for one character in a wider story; drawing an unwitting patsy in an attempt by a criminal gang to sell nuclear secrets which originated in the lab of (real life) nuclear proliferator A.Q. Khan. We liked this because it gave us a rich, fantastic backstory to work with, good guys and bad guys and a compelling target/goal for both teams to aim for.

Characters/Players

The game was played by 4 characters; essentially broken down into 2 teams - conspirators helping to smuggle nuclear secrets, government agents trying to stop this from happening. The characters were:

The Patsy - a dupe, drawn into the story through their own misfortune. The patsy is told they have won a prize, they just need to do a simple task before collecting their prize. Each time they do something - there’s just a little bit more demanded of them. This character was played by Michael Cohen.

The Exit Man - a buyer and mover of illicit items. The exit man will act as a contact for the patsy, and just wants to get the item so he can collect his reward. This character was played by Nara Kasbergen.

Operative - a low-level undercover agent tasked with gathering information, shadowing the Patsy. This character was played by Spike McCue.

The Handler - a senior agent who is given the full backstory of the plot, controls the actions of the operative and serves as the key decision maker for the “good guys”. This character was played by Eric Hagan.

In addition to the player characters there were two non-player characters; a Contact for the Sudanese criminal organisation directing the actions of the Exit Man and the Patsy, and a Senior Agent working for a government agency directing the actions of the Handler and the Operative. These characters were played by me and my super-talented classmate Natalie Be’er.

Back Story (Some Reality, Sprinkled with Fiction)

No single individual has done more to aid the spread of nuclear weapons than A.Q. Khan.

Khan was born in Pakistan in 1936 and received degrees in physics, mathematics and metallurgy. Khan’s 1972 doctoral thesis proposed several key improvements to the design of gas centrifuges; they key equipment involved in the enrichment of weapons’ grade uranium. Khan went on to hold key positions at engineering companies in Europe where he worked almost exclusively on the design of uranium enrichment systems.

It is now known that while working in Europe through the 1970s and early 1980s Khan made illicit copies of practically every document, drawing and computer file related to the production of highly enriched uranium. He developed relationships with a network of manufacturers and engineering companies around the world with whom he shared these documents; in essence creating a supply chain able to deliver nuclear weapons technology on demand. He worked closely with nuclear scientists in his native Pakistan to develop that country’s first viable nuclear weapon - the existence of which remained a secret until the mid 1980s. In 2003, under growing international pressure, Khan was questioned about the sale of nuclear technology. He confessed to having supplied designs to Libya, Iran and North Korea.

On his arrest, several tonnes of documents and drawings were seized from Khan’s lab - as well as 12 computers, dozens of external computer hard drives and prototype centrifuge parts. When a thorough inventory of these materials was produced an interesting pattern was spotted; key steps in the process of producing a viable nuclear weapon were completely missing. We believe that Khan managed to evacuate these documents several weeks in advance of the raid.

For 6 years we have had no credible intelligence as to the whereabouts of these documents; until now. Once of our sources has indicated that a Sudanese businessman Mr. Aziz Taha, acting on information from inside the Khan network, has located, or is close to locating these materials.

Little is known about Mr. Taha. He was certainly born in the Sudan in or around 1950. He was educated privately, and by the time he was in his early 20s had assumed a role at Sudan’s largest zinc extraction company. In 1985 his name appears in transcripts from a corruption trial involving the Sudanese energy minister; one witness describes Taha as a “talented man [...] able to solve import/export problems involving [...] known for his temper”. The government minister was subsequently acquitted following the disappearance of key witnesses. The man who mentioned Taha’s name was later found dead. This is the last official mention of Taha in any Sudanese record.

Taha became a subject of interest for international law enforcement agencies in 1993 when a forged shipping manifest linked one of his companies registered in Angola to a Soviet arms dealer Victor Anatolyevich Bout. It is probable that Taha was acting as a fixer for the illegal export of rare metals from Angola - using Bout as a courier. Another possible scenario is that Taha was acting as a local contact for an armed group being supplied with weapons by Bout. It may be that a two-way exchange of metals and weapons was involved - we simply do not know. There was insufficient evidence to arrest, or even detain Mr. Taha and he quickly disappeared.

We do not need to tell you how important it is that the Khan materials not fall into Taha’s hands; it is highly likely that he would quickly move to copy and sell them, or use them as leverage is some larger scheme.

Your job this week is to stop Taha from obtaining the Khan materials.

Game Play

The game played out over 5 days following a day by day playbook written by myself and Natalie. The playbook broke down the key plot points for the different characters, interventions that would need to be made by the non-player characters and any tools/props/materials we’d need to create.

Over the course of the week, several chapters didn’t go to plan or to schedule - so we essentially had to tweak the storyline in response to the actions of the characters; this was a lot of work, but also a lot of fun. For example, the Patsy didn’t pick up a set of customs documents in time forcing the Sudanese gang contact to reveal the true nature of the scheme - implicating the Patsy, drawing him closer into the plot.

The closing scenes of the game were initially supposed to involve a mobile phone hidden in the Strand bookstore, but we chose instead to introduce an encrypted message to all the characters, which if broken would lead them to grand central station in New York where the nuclear secrets were to be delivered; the decrypted message directed them to a train where we waited with a package (a small prize).

One team managed to crack half of the message, but our story finished with the nuclear secrets remaining on the train, returning to their destination; (maybe to re-appear at some point in the future). Just another package on a train.

Tools

Using the micro-film archives available at NYU we took real articles from the New York times which chronicled the development of A.Q. Khan’s proliferation network over 25 years. These were used to tell the back-story of our game to the characters. We then wrote and added layers of fictional details on top of these real events (the Sudanese gang etc.)

We used two Gmail and Google Voice accounts to communicate with both sides of the conspiracy over the course of the week; we could send emails and text messages to move the play along, change the story, give feedback/encouragement, pass on new information. We bought a cheap mobile phone for use in one of the chapters; but this section was (sadly) written out when play went in a different direction.

We tried to make the touch-points as real as possible; for example, in the chapter where the Patsy was supposed to fill in customs documents, we used real (though slightly modified) US customs documents.

Lessons Learned

The key things I learned were:

• It’s possible to write a compelling game that blends fact and fiction, real-life and technology - people are willing to suspend disbelief if the action is compelling enough.
• Blending real events/themes with fictional events gives you a really rich palette of material to draw from.
• A day by day playbook is a very good idea, but you have to allow flexibility - when things don’t go to plan, you have to have a fallback that doesn’t involve stopping the game/breaking character.
• With a small number of players and a reasonably simple story it’s possible to write for these contingencies on the fly; in fact this might even be better.
• People do not have a huge amount of time to play a game like this; they *might* go out of their way to travel great distances, overcome serious friction - but it’s also possible to mix up their daily routine just a little bit, ask them to do simple things to add depth; you don’t need to send them to faraway fields to dig up treasure for it to be fun.
• There is so much technology at your disposal to tell stories like this; the use of Gmail and Google voice really helped us to bring the game directly to the players.
• With more time you could craft communications tools, materials, props, non-player character identities really carefully; there’s probably no limit to the level of detail you could bring to a game like this.
• Writing and managing a game like this is a huge amount of work - and just as much fun!

All in all I hugely enjoyed designing, writing and executing this game - I’ll take the lessons I learned into my next projects.

I’ve never really been a fan of earthquake maps - aside from aberrations, over time they essentially plot the outline of the earth’s tectonic plates and fault lines. For me, there is a much closer mapping between earthquakes and sound waves, so I’ve tried to represent the last 30 days of earthquakes as sound.

In all cases, I am taking data from the USGS Shakemap feed at http://earthquake.usgs.gov/earthquakes/shakemap/rss.xml and using Processing to send MIDI into GarageBand, then exporting the resulting file as a waveform. Also, while the earthquakes are not evenly spaced in time - the notes I’m producing are. For now I’m more interested in depth and magnitude of the earthquakes mapping to pitch and volume; once I’ve solved those problems I can move on to the representation of time.

Challenges, What I Learned

These experiment really highlight how careful you need to be when mapping data to another form, or another scale. In the case of the earthquake data, the Richter scale is a logarithmic scale where each whole number is 10 times greater than the previous one; an earthquake measuing 9.0 is 10 times more powerful than one measuring 8.0 and so on. In MIDI, the volume of a note ranges from 0-127. In the experiments where I map straight from a rounded integer of the Richter measurement to MIDI the result makes smaller earthquakes sound much closer in force to larger ones.

Also, the experiment showed me that simultaneous, dramatic changes in pitch and volume are hard to process - I found myself listening to changes in pitch, or listening to changes in volume; but trying to keep track of both at the same time was almost too much information at the same time. Something I’ll keep in mind in later experiments.

Single Chord - Simple Depth and Volume Mapping

My first experiment; the depth of the earthquake in the Earth’s crust is mapped to the pitch of the note, and its Richter measurement is mapped to the Midi volume (problematic).

  30 Days of Earthquakes as a Single Chord by paulmmay

Simple Depth and Volume Mapping

In this piece of audio, the notes are evenly spaced - with the depth of the earthquake in the Earth’s crust is mapped to the pitch of the note, and its Richter measurement is mapped to the Midi volume (problematic).

  30 Days of Earthquakes - Depth Mapped to Pitch, Magnitude Mapped to Volume by paulmmay

Single Note, Simple Volume Mapping

I removed the pitch element which represented the depth of the earthquake; mostly because the change in pitch is far more jarring than the change in volume, but isn’t as important from a human perspective (given the Richter scale is a measurement of seismic energy).

In this sample, the Richter value is mapping straight to the 0-127 MIDI volume scale - which produces really problematic effect; effectively ignoring the logarithmic property of the Richter scale and making all earthquakes sound much closer in magnitude.

  30 Days of Earthquakes - Single Note, Magnitude Mapped to Volume by paulmmay

Single Note, Somewhat Logarithmic Volume

In this sample I am mapping the Richter value to a somewhat logarithmic value for the volume. For each point on the Richter scale from 9.0 to 0.0, the maximum volume of the note halves. This gives a spread of volumes that is discernable by the human ear, and somewhat solves the problem of weaker earthquakes sounding similar to more powerful ones.

  30 Days of Earthquakes - Fake Log Volume by paulmmay

Update - 15th March 2011

Thanks for the positive feedback on this work. I’ve done two more experiments; in both of these samples the timing of the earthquakes is relative whereas before they were all evenly spaced. I’ve compressed the time by a factor. The volume of the note also now uses a reasonably accurate mapping from the logarithmic Richter scale to the note’s volume.

In the first I’ve mapped the depth of the earthquake to the pitch of the piano note.

  30 Days of Earthquakes as Sound - Piano (Final) by paulmmay

In this sample I have stripped out the depth and am just using a single drum beat.

  30 Days of Earthquakes as Sound - Single Drum (Final) by paulmmay

For now, these are the last experiments using earthquake data. I’ve learned a lot, and it’s helped me get my head around the unfolding catastrophe in Japan in my own small way.