The Perpetual Horizon: Bodily self-perception in altered temporal perception
Associated body of work: Time Pieces
Salton Sea Revisited: an aesthetic study of realtime lapse.
(Reproduced from Proceedings of the 2013 IEEE VIS Arts Program and Leonardo Journal of International Society for the Arts, Sciences and Technology. Issue 49 Vol. 2 Spring 2015.)
Philosopher Vilem Flusser puts forward a hypothesis of two events that have changed the course of civilization: first the invention of linear type which moved us away from a cyclical experience of time to a linear, chronological experience, and second, the invention of the technical image which allows the recollection of time and memory through space and anachronistically [1]. I have added a third event, perhaps a subset of Flusser’s second point: the simultaneous experience of different scales of space and time, from nano to galactic and beyond. My particular focus is on scale and being able to shift the perception of the body through different scales and temporal experiences of space.
Fig. 1. Salton Sea Revisited. 6 March 2012. 0600—1800.
1. Architectural Body
Through the technical image our perceived body and the perception we have of the time and space we occupy are altered. Being able to shift the perceived boundaries of our body is the first paradigm for becoming what Madeline Gins and Arakawa call ‘architectural bodies’. An architectural body is an ‘organism-person-environment’, where through the perceptual dispersion of oneself through the environment we construct meaning. Tentativeness of this experiential constructing leads to a body in flux, and detached from Cartesian space.
Against the environment of the new territory that is her extended I, a person throws tentatives that land as functions and schemata, most of which join up with her, becoming of her by reprogramming her. Although the organism-person has the potential to become a person, it does not necessarily become one, or remain one. Everything begins for these organisms with a tentative constructing toward a holding in place. The environmental communal, which has everything to do with how an organism persons, can, when reworked in a concerted manner, lead to persons being able to supersede themselves [2].
What I am specifically aiming for in my general body of work is this last point: being able to supersede oneself through perception. Subjectivity resides beyond the point where proof exists. The study of perception is purely subjective and the epistemology is gained through varying ontologies making it incapable of being proven. Salton Sea Revisited and accompanying works focus on a very subjective and specific empirical knowledge: perception of self through perception of time and space. However, the result is not purely subjective but seeks realization and actuation in our current cultural, industrial and technological society.
With the aim of studying the limits of removing the body from experience—the two are intertwined—how will new aesthetic techniques and technological innovations of altering perceptions of self, time and space arise? The reverse is also worth studying: How do we create new experiences that induce a sense of dematerialization? The study of the perceptual qualities of time and space will connect with studies of sustainability, mindfulness, innovation and forward thinking once we learn how much rides on perception and how it can be teased out and altered for the greater good. This body of work looks at this very topic of perception of time and space to redefine architecture to a utopian and body-centered environment.
2. Realtime Lapse
“The only reason for time is so that everything doesn't happen at once.”—Albert Einstein
Everything does happen at once: at every given moment almost all imaginable things are happening for billions of living beings, at once. What does not happen at once is our experience of a moment in time.
The first image that altered my perception, inducing an awe such that my course of thinking about my work shifted drastically to revisit my philosophy and aesthetics, is from the Hubble Ultra Deep Field, an image of the universe in infancy, 13.7 billion years ago [3]. What we see is time travel (I am a science fiction fan), and most importantly where we see this from is at a universal scale of our body: by holding the image of a space so great in magnitude that it is incomprehensible if we were at the same scale, the image becomes accessible by nature of perceiving ourselves to be greater in scale.
Through the technical image, not only do we see different scales of space, we experience different scales of our body. Can we assume our scale has any effect in how we experience time in different space scales? Can we be experiencing time as we are now––second by second––because of our small scale in relation to the space our time is in relation to, a space in which we are very small? Can we be infinitely huge and experience all of space and its time at once? What happens to our perception of reality, of space and of self, when we are able to experience time simultaneously?
The answer to these questions is what I set out to explore with my photography and video and the specific editing technique of slicing time and space to be experienced at once. The video and photography show the entire span of a day––usually from sunsight to sunclipse––being experienced at once. This simultaneous experience of time is akin to our experience of first seeing images of Earth—the sudden awe in seeing the massive scale of the whole. The activation of awe is key. Consequent studies show that awe expands the perception of time and involves perceptual vastness that arises ‘when one encounters something so strikingly vast that it provokes a need to update one’s mental schemas… meaning that it alters one’s understanding of the world’ [4]. In these images, I alter time and space to induce awe, to in return alter the perception of the body. The expanded moment leads to a feeling of psychological and physical expansion in and across space. Different from the referenced experimental psychology studies, I take the relationship between time and awe as a given for experiencing beauty of the extraordinary, or the unusual. Awe is necessary towards becoming architectural bodies.
Fig. 2. Salton Sea North Shore. 18 January 2010. 0600—1800.
Prior to my first image [Fig. 2], this technique of simultaneous spatial and temporal exploration had not been explored in this particular aesthetic, therefore naming this technique became a challenge in understanding the information the image puts forth. During 2009-2010 erroneously I called these images time-lapses, however, in a traditional time-lapse, time is continuous and only sped-up. Here time lapses in chunks, literally, a lapse, missing information both in time and landscape over time. By 2010-2011 I chose time slices, however in video, the nature of time and landscape lapsing is lost through the choice of ‘slice’. In 2012, I settled on real-time time-lapse and finally realtime lapse to stress the real passage of time, though lapsing across the landscape in time. A recent feature of the work on the popular, curated data-visualization blog, Flowing Data, Nathan Yau referred to the work as ‘time running parallel’ [5]. Though intriguing, parallel time implies parallel space, in other words, different spaces that are adjacent, whereas the space in these videos and images are the same. This is a topic that needs to be further explored.
3. Aesthetic Experiments of Time Perception
A second challenge was determining the temporal length of the video and the image. In bands that are too long in duration or too wide in pixel, a sense of the moment, or perceptual present is lost. The perceptual present is the moment where information is collected, requiring no recollection of memory of the events that have taken place; it is the Now [6]. Studies report the duration of the present to be between 2 to 8 seconds, with 3 seconds being the most common reported [7]. To experience times across the day as one, simultaneous time event, the viewer must be constantly kept in the perceptual present and in the indifference interval, where underestimation or overestimation of time does not occur [8]. This also draws on what Stockhausen called moment form, ‘where a piece unfolds as a succession of unrelated moments’ [9]:
It has to be the degree of immediacy, of presence that unites the individual moments: the fact that everything has presence to the same degree, because as soon as certain events are more present than others, then immediately we have a hierarchy.
Tests of duration were done in 1 minute, 2, 3 and 5 minute lengths to learn which offers the best perception of repeated perceptual presents so that the viewer perceives many ‘times’ as one. This is dependent on the context of the scene and in a moderately paced environment a 3 to 3.5 minute length is ideal where the events of the scene fall within the 2-8 second passage. In the sub-challenge of widths for photographs, I was more concerned with how much movement is required by the eye to collect information before each slice of time was seen as a separate image as opposed to perceived as a continuation of the slices on either side of it. These aesthetic experiments draw on foundational Gestalt theory in both sound and visual realms and are very subjective and varied from one image or video to another. An average still image is between 40 and 50 slices from sunsight to sunclipse; the video varies depending on context, as explained above.
4. Aesthetic Discoveries of Data and Events
Two aesthetic discoveries have emerged from my video and photographic technique. First, the emergence of data not perceivable in a moment-by-moment based experience of time. The first successful image was compiled in January 2010 of a stormy day at the Salton Sea [Fig. 2]. It produced a striking visualization of the weather pattern throughout the day. Given that the Salton Sea is a major wetland for over 400 species of migrating birds from Eastern Russia to South America, subsequent images [Fig. 1] became rich visualizations of wildlife activity and patterns of behavior. Additional images in other locations [Fig. 3] showed dynamic movement of the landscape over time through one static image, a swifter visual data gathering and information processing method than looking at multiple, full-scene images side-by-side where our cognitive process is forced to parse useless information gathered by our vision. A recent exhibit focusing on air pollution in Los Angeles over the course of the month uses this technique, as well as another project that simply shows South Bank Centre in London [10, 11].
Fig. 3. Malibu Lagoon. 3 December 2012. 0659—1649 / 0659—1139 / 0659—1139. Three different time slices with different aesthetic and perceptual results.
The second discovery, and one more in line with my research goals of becoming architectural bodies and moving beyond the limits of the body, is that insignificant moments become significant events through the simultaneous experience of time. In Cinema 1: The Movement-Image, Gilles Deleuze defines every image as a set of rarefied or saturated data [12]. Through the rarefication of the landscape by cutting it up across time and eliminating most of the information, I get a saturation of information. So the image does not hold one or the other form of data, but both. This saturation across time that is experienced at once leads to moments becoming events. What is an event but a significant moment? Events expand the moment; this expansion leads to awe; awe alters our perception of time. Time therefore becomes limitless.
In a sense of limitless time we become infinite.
/End
Addendum I: Moment-Event as an aesthetic descriptor in video
In the above paper published in the Leonardo Journal (49:2) I touch on the finding of moments becoming events as aesthetic information coming from a saturation of data. In my research focus on the perception of time, this is conceptually the same as time in a given moment expanding to hold an event. In other words, a point holds duration and the perception of time is prolonged. But what escaped me was identifying this finding as a concise aesthetic descriptor for key moments in a film or video, termed Moment-Event. To visualize, Moment-Event is seen between 2:13 and 2:18 in the video Salton Sea Revisited . This is only possible in the method of editing the scene as a realtime lapse.
The first instance I used this terminology was in design notes on a project I am developing where I was listing elements in a particular video scene that were needed in order to develop a narrative. Among listing cardinal directions and landmarks in the landscape, Moment-Event captured precisely and concisely what would have previously required many sentences and metaphors and similes to describe. It only requires a quick example (given above) and a concept greater than what can be described is encapsulated in two words: Moment-Event.
Addendum II: Realtime lapse as a model of small multiples visualization for qualitative expression and construction of knowledge
To further develop a section of my IEEE Vis paper, Salton Sea Revisited: An Aesthetic Study of Real-time Lapse (2013), I expand on the aesthetic technique of real-time lapse. Realtime lapse is an aesthetic technique of simultaneous time slices, allowing the emergence of data not perceivable in a moment-by-moment based experience of time. In particular, unique to the method of real-time lapse is the use of video. The result is a new model expanding the language of Edward Tufte’s static small multiples (‘frames of a movie’) and incorporating Johanna Drucker’s expression of capta as opposed to visualization of data, to simultaneously capture qualitative information of an environment that can lead to further quantifiable studies and construction of knowledge.
As data visualization has crossed the scientific realm of quantifying knowledge into the realm of artistic representations of the flow and aggregation of information, I look into the method of real-time lapse as a model for constructing knowledge from an aesthetic standpoint. Whereas the common perception is of the scientific methods yielding intended knowledge and unintended aesthetic results, here the reverse is true where the purely aesthetic and conceptual method drawing on qualitative ontology can return quantifiable epistemology.
Recent research has delved into studies on animation and small multiples, with experiment results conducive to the theory that animations do indeed increase the rate of information absorption both cognitively and visually, as well as taking less time to analyze the given data. No experiments will be presented in this paper, but suggestions will be left open for further research based on provided information on frame rate and width of conjoined space-time narratives within standard video frame sizes.
In all papers studied on animation and small multiple visualizations in the temporal domain, surprisingly, not one took a simply literal stance on Tufte’s definition of small multiple to see what it looks like, what does it mean, is it possible, or how could it be possible:
“Small multiples resemble the frames of a movie: a series of graphics, showing the same combination of variables, indexed by changes in another variable.” Theory of Data Graphics. p. 170
Although Tufte’s work was not at the forefront of my research, the underlying questions are similar: How to simultaneously show changes in time, a dimension we experience moment by moment? Additionally, I am interested in knowing what the aesthetic result is? What will be gained from this way of seeing? What new experiences will come from this method? In particular, how does this way of experiencing time and space affect our bodily self-perception in regards to our perceived scale of our body?
Figure X shows 18 frames (1 frame every 10 minutes) from video taken on 25 September 2012 at Malibu Lagoon from 7am to 12pm, shown as small multiples. Here, as is expected of the theory of small multiples, adequate spacing in time and difference in each sample allows for an overall quick judgment of the result of the changes between ‘frames’: a foggy morning leading to a lightly overcast mid-morning and slowly parting clouds at mid-day. However, dependent on the context, changes and differences in such a short time span can easily go unnoticed or difficult to recognize, for example, the slight vibrancy changes in the early morning fog.
Figure XXa shows the same selected frames (1 frame every 10 minutes) in slices. The Malibu Lagoon realtime lapse (Figure XXa) depicts an even wildlife activity and the dissipation of fog to cloud, but great variation in the color and diffusion of light, a quality that goes unnoticed or barely noticed in the small multiples model (Figure X). In comparison, in the Salton Sea realtime lapse (Figure XXb), we can quickly grasp wildlife activity—sunrise and sunset being quite active with the most diverse bird species, and noon time an additional active feeding time for larger bird species. In both instances, a general quality of the environment can be extrapolated.
In this model of representation, the landscape that is constant and unchanging (the jette at Salton Sea, the sandbar at Malibu Lagoon) is reconstructed with slices of the landscape across time. This allows for the simultaneous passage of time across the landscape. But by slicing the images, the edit line introduces what Delueze in Cinema 1 calls ‘privileged instances’: The nature of the edit line places attention on moments of potentialities that are fleeting and passing in the single-frame point of view of small multiples. However, these privileged instances only become apparent in motion, not as stills, which are merely a succession of instances. This is the significant improvement and advantage over static small multiple frames or abstract short-duration animations because, regardless of tying to Gestalt theories of prägnanz, motion is a foundation of curiosity and scientific research.
Figure XXX shows two extrapolations of Figure XXa, with wider slices in the same time frame, and where Figure XXXb is 10 minutes ahead of Figure XXXa.
Griffin’s animation:
“Another purpose for which animated representations might be used is for knowledge construction and discovery while viewers are exploring data. In this case, the viewer’s primary goal might be to notice a pattern in his or her data, rather than (at least initially) trying to understand the process that is creating that pattern. Our goal in this experiment was to investigate the relative effectiveness of animated and static small-multiple representations of space-time information in aiding viewers with knowledge construction and discovery tasks.” P.2
No study prior to my work has looked at simultaneous realtime video and its effect. The advantage to my method is that changes across a large amount of time can be studied in real-time, as opposed to the compressed time of conventional time-lapses, or abstractions of an animation, or any method that eliminates large amounts of time.
References
[1] Vilem Flusser. Towards a Philosophy of Photography. Reaktion Books. p. 7. 2000.
[2] Madeline Gins & Arakawa. Architectural Body. University of Alabama Press. pp. 46-7. 2002.
[3] NASA Hubble Ultra Deep Field. Retrieved 6 September 2013. http://www.nasa.gov/mission_pages/hubble/science/farthest-galaxy.html
[4] Melanie Rudd, Vohs, K. D., & Aaker, J. Awe Expands People's Perception of Time, Alters Decision Making, and Enhances Well-Being. Psychological Sciences 23(10) pp. 1130-1136. 2012.
[5] Nathan Yau. Retrieved 6 September 2013. http://flowingdata.com/2013/02/01/time-running-parallel/
[6] Eric F. Clarke. Rhythm and Timing in Music. In The Psychology of Music, ed. Diana Deutsch: pp. 473-500. San Diego: Academic Press. 1999.
[7]Pöppel, Ernst. Lost in Time: A Historical Frame, Elementary Processing Units and the 3-second Window. Acta Neurobiol Exp 64: pp. 295-301. 2004.
[8] Eric F. Clarke. Rhythm and Timing in Music. In The Psychology of Music, ed.
[9] Diana Deutsch: pp. 473-500. San Diego: Academic Press. 1999.
[10] Jeff Rau. June 2011-May 2012. Retrieved 6 September 2013. http://www.jeffrau.com/art/artwork/haze-project/
[11] Curtis Roads. Composing Electronic Music: A New Aesthetic. Oxford University Press. Forthcoming.
[12] James Bridle. 1 June 2012. Retrieved 6 September 2013. http://booktwo.org/notebook/a-ship-aground/
[13] Gilles Deleuze. Cinema 1: The Movement Image. p12. University of Minnesota Press. 1986.
(Reproduced from Proceedings of the 2013 IEEE VIS Arts Program and Leonardo Journal of International Society for the Arts, Sciences and Technology. Issue 49 Vol. 2 Spring 2015.)
Philosopher Vilem Flusser puts forward a hypothesis of two events that have changed the course of civilization: first the invention of linear type which moved us away from a cyclical experience of time to a linear, chronological experience, and second, the invention of the technical image which allows the recollection of time and memory through space and anachronistically [1]. I have added a third event, perhaps a subset of Flusser’s second point: the simultaneous experience of different scales of space and time, from nano to galactic and beyond. My particular focus is on scale and being able to shift the perception of the body through different scales and temporal experiences of space.
Fig. 1. Salton Sea Revisited. 6 March 2012. 0600—1800.
1. Architectural Body
Through the technical image our perceived body and the perception we have of the time and space we occupy are altered. Being able to shift the perceived boundaries of our body is the first paradigm for becoming what Madeline Gins and Arakawa call ‘architectural bodies’. An architectural body is an ‘organism-person-environment’, where through the perceptual dispersion of oneself through the environment we construct meaning. Tentativeness of this experiential constructing leads to a body in flux, and detached from Cartesian space.
Against the environment of the new territory that is her extended I, a person throws tentatives that land as functions and schemata, most of which join up with her, becoming of her by reprogramming her. Although the organism-person has the potential to become a person, it does not necessarily become one, or remain one. Everything begins for these organisms with a tentative constructing toward a holding in place. The environmental communal, which has everything to do with how an organism persons, can, when reworked in a concerted manner, lead to persons being able to supersede themselves [2].
What I am specifically aiming for in my general body of work is this last point: being able to supersede oneself through perception. Subjectivity resides beyond the point where proof exists. The study of perception is purely subjective and the epistemology is gained through varying ontologies making it incapable of being proven. Salton Sea Revisited and accompanying works focus on a very subjective and specific empirical knowledge: perception of self through perception of time and space. However, the result is not purely subjective but seeks realization and actuation in our current cultural, industrial and technological society.
With the aim of studying the limits of removing the body from experience—the two are intertwined—how will new aesthetic techniques and technological innovations of altering perceptions of self, time and space arise? The reverse is also worth studying: How do we create new experiences that induce a sense of dematerialization? The study of the perceptual qualities of time and space will connect with studies of sustainability, mindfulness, innovation and forward thinking once we learn how much rides on perception and how it can be teased out and altered for the greater good. This body of work looks at this very topic of perception of time and space to redefine architecture to a utopian and body-centered environment.
2. Realtime Lapse
“The only reason for time is so that everything doesn't happen at once.”—Albert Einstein
Everything does happen at once: at every given moment almost all imaginable things are happening for billions of living beings, at once. What does not happen at once is our experience of a moment in time.
The first image that altered my perception, inducing an awe such that my course of thinking about my work shifted drastically to revisit my philosophy and aesthetics, is from the Hubble Ultra Deep Field, an image of the universe in infancy, 13.7 billion years ago [3]. What we see is time travel (I am a science fiction fan), and most importantly where we see this from is at a universal scale of our body: by holding the image of a space so great in magnitude that it is incomprehensible if we were at the same scale, the image becomes accessible by nature of perceiving ourselves to be greater in scale.
Through the technical image, not only do we see different scales of space, we experience different scales of our body. Can we assume our scale has any effect in how we experience time in different space scales? Can we be experiencing time as we are now––second by second––because of our small scale in relation to the space our time is in relation to, a space in which we are very small? Can we be infinitely huge and experience all of space and its time at once? What happens to our perception of reality, of space and of self, when we are able to experience time simultaneously?
The answer to these questions is what I set out to explore with my photography and video and the specific editing technique of slicing time and space to be experienced at once. The video and photography show the entire span of a day––usually from sunsight to sunclipse––being experienced at once. This simultaneous experience of time is akin to our experience of first seeing images of Earth—the sudden awe in seeing the massive scale of the whole. The activation of awe is key. Consequent studies show that awe expands the perception of time and involves perceptual vastness that arises ‘when one encounters something so strikingly vast that it provokes a need to update one’s mental schemas… meaning that it alters one’s understanding of the world’ [4]. In these images, I alter time and space to induce awe, to in return alter the perception of the body. The expanded moment leads to a feeling of psychological and physical expansion in and across space. Different from the referenced experimental psychology studies, I take the relationship between time and awe as a given for experiencing beauty of the extraordinary, or the unusual. Awe is necessary towards becoming architectural bodies.
Fig. 2. Salton Sea North Shore. 18 January 2010. 0600—1800.
Prior to my first image [Fig. 2], this technique of simultaneous spatial and temporal exploration had not been explored in this particular aesthetic, therefore naming this technique became a challenge in understanding the information the image puts forth. During 2009-2010 erroneously I called these images time-lapses, however, in a traditional time-lapse, time is continuous and only sped-up. Here time lapses in chunks, literally, a lapse, missing information both in time and landscape over time. By 2010-2011 I chose time slices, however in video, the nature of time and landscape lapsing is lost through the choice of ‘slice’. In 2012, I settled on real-time time-lapse and finally realtime lapse to stress the real passage of time, though lapsing across the landscape in time. A recent feature of the work on the popular, curated data-visualization blog, Flowing Data, Nathan Yau referred to the work as ‘time running parallel’ [5]. Though intriguing, parallel time implies parallel space, in other words, different spaces that are adjacent, whereas the space in these videos and images are the same. This is a topic that needs to be further explored.
3. Aesthetic Experiments of Time Perception
A second challenge was determining the temporal length of the video and the image. In bands that are too long in duration or too wide in pixel, a sense of the moment, or perceptual present is lost. The perceptual present is the moment where information is collected, requiring no recollection of memory of the events that have taken place; it is the Now [6]. Studies report the duration of the present to be between 2 to 8 seconds, with 3 seconds being the most common reported [7]. To experience times across the day as one, simultaneous time event, the viewer must be constantly kept in the perceptual present and in the indifference interval, where underestimation or overestimation of time does not occur [8]. This also draws on what Stockhausen called moment form, ‘where a piece unfolds as a succession of unrelated moments’ [9]:
It has to be the degree of immediacy, of presence that unites the individual moments: the fact that everything has presence to the same degree, because as soon as certain events are more present than others, then immediately we have a hierarchy.
Tests of duration were done in 1 minute, 2, 3 and 5 minute lengths to learn which offers the best perception of repeated perceptual presents so that the viewer perceives many ‘times’ as one. This is dependent on the context of the scene and in a moderately paced environment a 3 to 3.5 minute length is ideal where the events of the scene fall within the 2-8 second passage. In the sub-challenge of widths for photographs, I was more concerned with how much movement is required by the eye to collect information before each slice of time was seen as a separate image as opposed to perceived as a continuation of the slices on either side of it. These aesthetic experiments draw on foundational Gestalt theory in both sound and visual realms and are very subjective and varied from one image or video to another. An average still image is between 40 and 50 slices from sunsight to sunclipse; the video varies depending on context, as explained above.
4. Aesthetic Discoveries of Data and Events
Two aesthetic discoveries have emerged from my video and photographic technique. First, the emergence of data not perceivable in a moment-by-moment based experience of time. The first successful image was compiled in January 2010 of a stormy day at the Salton Sea [Fig. 2]. It produced a striking visualization of the weather pattern throughout the day. Given that the Salton Sea is a major wetland for over 400 species of migrating birds from Eastern Russia to South America, subsequent images [Fig. 1] became rich visualizations of wildlife activity and patterns of behavior. Additional images in other locations [Fig. 3] showed dynamic movement of the landscape over time through one static image, a swifter visual data gathering and information processing method than looking at multiple, full-scene images side-by-side where our cognitive process is forced to parse useless information gathered by our vision. A recent exhibit focusing on air pollution in Los Angeles over the course of the month uses this technique, as well as another project that simply shows South Bank Centre in London [10, 11].
Fig. 3. Malibu Lagoon. 3 December 2012. 0659—1649 / 0659—1139 / 0659—1139. Three different time slices with different aesthetic and perceptual results.
The second discovery, and one more in line with my research goals of becoming architectural bodies and moving beyond the limits of the body, is that insignificant moments become significant events through the simultaneous experience of time. In Cinema 1: The Movement-Image, Gilles Deleuze defines every image as a set of rarefied or saturated data [12]. Through the rarefication of the landscape by cutting it up across time and eliminating most of the information, I get a saturation of information. So the image does not hold one or the other form of data, but both. This saturation across time that is experienced at once leads to moments becoming events. What is an event but a significant moment? Events expand the moment; this expansion leads to awe; awe alters our perception of time. Time therefore becomes limitless.
In a sense of limitless time we become infinite.
/End
Contribution
Addendum I: Moment-Event as an aesthetic descriptor in video
In the above paper published in the Leonardo Journal (49:2) I touch on the finding of moments becoming events as aesthetic information coming from a saturation of data. In my research focus on the perception of time, this is conceptually the same as time in a given moment expanding to hold an event. In other words, a point holds duration and the perception of time is prolonged. But what escaped me was identifying this finding as a concise aesthetic descriptor for key moments in a film or video, termed Moment-Event. To visualize, Moment-Event is seen between 2:13 and 2:18 in the video Salton Sea Revisited . This is only possible in the method of editing the scene as a realtime lapse.
The first instance I used this terminology was in design notes on a project I am developing where I was listing elements in a particular video scene that were needed in order to develop a narrative. Among listing cardinal directions and landmarks in the landscape, Moment-Event captured precisely and concisely what would have previously required many sentences and metaphors and similes to describe. It only requires a quick example (given above) and a concept greater than what can be described is encapsulated in two words: Moment-Event.
Addendum II: Realtime lapse as a model of small multiples visualization for qualitative expression and construction of knowledge
To further develop a section of my IEEE Vis paper, Salton Sea Revisited: An Aesthetic Study of Real-time Lapse (2013), I expand on the aesthetic technique of real-time lapse. Realtime lapse is an aesthetic technique of simultaneous time slices, allowing the emergence of data not perceivable in a moment-by-moment based experience of time. In particular, unique to the method of real-time lapse is the use of video. The result is a new model expanding the language of Edward Tufte’s static small multiples (‘frames of a movie’) and incorporating Johanna Drucker’s expression of capta as opposed to visualization of data, to simultaneously capture qualitative information of an environment that can lead to further quantifiable studies and construction of knowledge.
As data visualization has crossed the scientific realm of quantifying knowledge into the realm of artistic representations of the flow and aggregation of information, I look into the method of real-time lapse as a model for constructing knowledge from an aesthetic standpoint. Whereas the common perception is of the scientific methods yielding intended knowledge and unintended aesthetic results, here the reverse is true where the purely aesthetic and conceptual method drawing on qualitative ontology can return quantifiable epistemology.
Recent research has delved into studies on animation and small multiples, with experiment results conducive to the theory that animations do indeed increase the rate of information absorption both cognitively and visually, as well as taking less time to analyze the given data. No experiments will be presented in this paper, but suggestions will be left open for further research based on provided information on frame rate and width of conjoined space-time narratives within standard video frame sizes.
In all papers studied on animation and small multiple visualizations in the temporal domain, surprisingly, not one took a simply literal stance on Tufte’s definition of small multiple to see what it looks like, what does it mean, is it possible, or how could it be possible:
“Small multiples resemble the frames of a movie: a series of graphics, showing the same combination of variables, indexed by changes in another variable.” Theory of Data Graphics. p. 170
Although Tufte’s work was not at the forefront of my research, the underlying questions are similar: How to simultaneously show changes in time, a dimension we experience moment by moment? Additionally, I am interested in knowing what the aesthetic result is? What will be gained from this way of seeing? What new experiences will come from this method? In particular, how does this way of experiencing time and space affect our bodily self-perception in regards to our perceived scale of our body?
Figure X shows 18 frames (1 frame every 10 minutes) from video taken on 25 September 2012 at Malibu Lagoon from 7am to 12pm, shown as small multiples. Here, as is expected of the theory of small multiples, adequate spacing in time and difference in each sample allows for an overall quick judgment of the result of the changes between ‘frames’: a foggy morning leading to a lightly overcast mid-morning and slowly parting clouds at mid-day. However, dependent on the context, changes and differences in such a short time span can easily go unnoticed or difficult to recognize, for example, the slight vibrancy changes in the early morning fog.
Figure XXa shows the same selected frames (1 frame every 10 minutes) in slices. The Malibu Lagoon realtime lapse (Figure XXa) depicts an even wildlife activity and the dissipation of fog to cloud, but great variation in the color and diffusion of light, a quality that goes unnoticed or barely noticed in the small multiples model (Figure X). In comparison, in the Salton Sea realtime lapse (Figure XXb), we can quickly grasp wildlife activity—sunrise and sunset being quite active with the most diverse bird species, and noon time an additional active feeding time for larger bird species. In both instances, a general quality of the environment can be extrapolated.
In this model of representation, the landscape that is constant and unchanging (the jette at Salton Sea, the sandbar at Malibu Lagoon) is reconstructed with slices of the landscape across time. This allows for the simultaneous passage of time across the landscape. But by slicing the images, the edit line introduces what Delueze in Cinema 1 calls ‘privileged instances’: The nature of the edit line places attention on moments of potentialities that are fleeting and passing in the single-frame point of view of small multiples. However, these privileged instances only become apparent in motion, not as stills, which are merely a succession of instances. This is the significant improvement and advantage over static small multiple frames or abstract short-duration animations because, regardless of tying to Gestalt theories of prägnanz, motion is a foundation of curiosity and scientific research.
Figure XXX shows two extrapolations of Figure XXa, with wider slices in the same time frame, and where Figure XXXb is 10 minutes ahead of Figure XXXa.
Griffin’s animation:
“Another purpose for which animated representations might be used is for knowledge construction and discovery while viewers are exploring data. In this case, the viewer’s primary goal might be to notice a pattern in his or her data, rather than (at least initially) trying to understand the process that is creating that pattern. Our goal in this experiment was to investigate the relative effectiveness of animated and static small-multiple representations of space-time information in aiding viewers with knowledge construction and discovery tasks.” P.2
No study prior to my work has looked at simultaneous realtime video and its effect. The advantage to my method is that changes across a large amount of time can be studied in real-time, as opposed to the compressed time of conventional time-lapses, or abstractions of an animation, or any method that eliminates large amounts of time.
References
[1] Vilem Flusser. Towards a Philosophy of Photography. Reaktion Books. p. 7. 2000.
[2] Madeline Gins & Arakawa. Architectural Body. University of Alabama Press. pp. 46-7. 2002.
[3] NASA Hubble Ultra Deep Field. Retrieved 6 September 2013. http://www.nasa.gov/mission_pages/hubble/science/farthest-galaxy.html
[4] Melanie Rudd, Vohs, K. D., & Aaker, J. Awe Expands People's Perception of Time, Alters Decision Making, and Enhances Well-Being. Psychological Sciences 23(10) pp. 1130-1136. 2012.
[5] Nathan Yau. Retrieved 6 September 2013. http://flowingdata.com/2013/02/01/time-running-parallel/
[6] Eric F. Clarke. Rhythm and Timing in Music. In The Psychology of Music, ed. Diana Deutsch: pp. 473-500. San Diego: Academic Press. 1999.
[7]Pöppel, Ernst. Lost in Time: A Historical Frame, Elementary Processing Units and the 3-second Window. Acta Neurobiol Exp 64: pp. 295-301. 2004.
[8] Eric F. Clarke. Rhythm and Timing in Music. In The Psychology of Music, ed.
[9] Diana Deutsch: pp. 473-500. San Diego: Academic Press. 1999.
[10] Jeff Rau. June 2011-May 2012. Retrieved 6 September 2013. http://www.jeffrau.com/art/artwork/haze-project/
[11] Curtis Roads. Composing Electronic Music: A New Aesthetic. Oxford University Press. Forthcoming.
[12] James Bridle. 1 June 2012. Retrieved 6 September 2013. http://booktwo.org/notebook/a-ship-aground/
[13] Gilles Deleuze. Cinema 1: The Movement Image. p12. University of Minnesota Press. 1986.