Talk outline for SXSW Glitch Panel and background material
1. Who am I? New media artist who works with virtual worlds, electronic
literature, and musical improvisation. I'm particularly interested in the
relationship of the digital "clean screen" and issues of abjection -
dirtiness, wounding, ecstasy, and distortions, of avatars and real flesh.
In other words, what lies beneath the surface.
2. I've used motion capture as a way of creating images, 3d printed
objects, virtual world avatars and performances, and live choreographies -
by distorting the mocap "chain" which goes from live performer to computer
file to an avatar representation based on that file. (Illustrate hand
gestures)
3. The chain can be altered in several ways:
a. by remapping motion capture nodes across the body. (Example)either
Frances.mpg (Illustrate hand gestures)
-- in other words software that's forced through glitches
b. by remapping mocap nodes across several bodies.
(Example) commodius.mp4 mocaps.mov 322harness.mp4 (w/still)
c. by reworking the mocap software itself so that it uses dynamic or
behavioral filters that transform the avatar representation by
transforming the files as they are collected. (Illustrate hand gestures)
(Example)
d. by moving the performers out to the edge of the game-space itself -
(Example) edge_6.avi - move cursor to relevant portion
4. The results can be:
distorted avatar movements in video / still images
(Example) WolfTC.mov sports.mov air.mov noOne.mov
live performances in Second Life and mixed reality media (Example)
anything in the SL folder
3d printed objects
(Example) 3d stills folder
choreographies with Foofwa d'Imobilite
(Example) dnaclass duetavatargrange involuntary
images, videos, anything
5. Why?
How to confound the digital with the abject - with issues of pain,
wounding, death, ecstasy.
How to dis-embed the embedded news media - in other words, how to reveal
the deaths of soldiers and civilians around the world, in spite of or
through digital media. Of course avatars can't do that, but they can
resonate with the slaughter that occurs in the real - rather than
constantly resonating with the virtual slaughter that occurs in video
games. Get rid of the radical disconnect.
1 digital is dirty:
potential wells, channel noise, surface hacking/cracking,
leaking through the real
2 digital is corporate: protocols and codecs, open source,
closed source, committees, communities, TAZ (temporary
autonomous zones)
3 digital tends towards eternity, infinity, closure
4 digital forecloses the analogic, dirty, abjection
5 glitch occurs through or within surfaces
6 glitch tends towards novelty
7 glitch like wounding is intermediary between the
accidental and the determinate-stochastic
8 glitch is dead-inert
9 the opposite of glitch may well be suturing
10 glitch: jump-cut, suturing: continuity-girl
11 analog: is dead-inert; glitch is the scarring of the
analogic
12 reality: neither analog nor digital, entangled
visual data representation - empathetic identifications - the mocap work
social experimentation - four dancers into one avatar for example
psychological experimentation - "reading" the modified avatar -
identified as human or organism?
identified as wounded or dancing?
GLITCH: 1, the use of glitch in virtual worlds and how it extends avatar
possibilities; 2, where glitch kills - errors which give glitch an abject
edge; 3, glitch in my motion-capture work; 4, psychoanalytics of glitch;
5, glitch and programming - 'not a bug but a feature.'
Difference between unutterable pain and its (external) representation, and
utterable programming of its (external, at a double remove)
representation.
DIRECTORIES: 3dstills/chic/dance/sl/wvu/ and other
1. Text > behavioral filtering, dynamic mocap.
- WVU transformation and dynamic/behavioral filtering
- Chicago four dancers into one: the avatar as social organism
2. Images of:
pain, disfigurement, torture, ecstasy, death
- in the virtual > Poser, Blender, Second Life, Open Sim
- in the real > unutterable pain, dance -
- return of structure to the real
Showing:
1. Mocap video from Chicago and West Virginia
2. Images directly produced from Mocap
3. Images modified and inserted into Second Life
4. Movements taken from Second Life back into the real
I'm interested in human representation within the virtual; this implies
both an image and a dynamics, the two of course entangled.
More to the point, though, has been the issue of movement. On and off for
the past two decades, I've worked with Foofwa d'Imobilite [give background
information here]. Foofwa's work is at foofwa.com. His work has dealt with
any number of issues, from politics through health, sexuality, being-
Swiss, technology to virtuality. We've embedded him in Second Life, and
he's worked with our avatar movement as well. *
Motion capture work: Brief history of my work in the Virtual Environments
Lab at West Virginia University, Morgantown, through Sandy Baldwin and
Frances Van Scoy.
1. We recorded one or more performers, using remappings of modes,
including some with 'impossible' topologies, in terms of human movement;
and
2. We recorded, through a rewriting of the motion capture software itself,
through what I've called 'dynamic filtering,' transforming standard motion
capture files on the fly, through the insertion of filters between the
input data and the outputted files. These filters parallel the use of
filters in Gimp or Photoshop [explain].
Endproducts - the altered motion capture files were fed into three worlds:
1. The Blender 3d modeling program, where abstract avatars were used to
examine how behavior appears when it's abstracted from the body;
2. The Poser mannequin modeling program, where the motion capture files
were used to 'break' the mannequin bodies, as well create any number of
videos; and
3. Works in Second Life and OpenSim virtual worlds which involved highly
distorted avatar performances and dances; these were used for live or
mixed reality performance, some augmented reality work, some video work
for conferences, gallery or museum installations, and some pieces made for
live or online choreographies. The ultimate goals of the virtual worlds
work were - what happens when the body is considered completely plastic;
what images of pain, death, wounding, or sexuality are conjured up by
distorted bodies; when does the body become a 'thing' among other things
in the world; what are the politics and anthropology of distorted avatars
and movements - if any.
Last year, Patrick Lichty enabled me to use the highly sophisticated
motion capture equipment at Columbia College, Chicago; here, we didn't
modify any software (we had neither the expertise nor permissions!);
instead, we worked closely with remapping the body in relation to the
30-40 markers that were placed on the body suits. This is where everything
becomes interesting, I think, since we were able to map up to four
dancers/performers into a single avatar output. It was difficult to do
this because the software tended to stop working and 'glitch' the avatar
into a somewhat inert Buddhist image when it could no longer make sense of
the input. But we were able to create complex movements, and one technique
stood out - the 'hive' technique or social avatar 2.0.
The usual mappings we did involved a single performer with the body nodes
remapped on him or her. So there was a topology involved; the hip was
usually the stable or root node. In West Virginia, we started using two
performers; this is what can happen: [demo the torsion/twist]. When I was
in Chicago, I was able to work with four performers, two on trapezes, all
choreographed into a single avatar - and all capable of watching the
results of their movement on a screen. So we tried:
1. Moving the avatar in utterly untoward ways, so that the result was a
limping or broken avatar; and
2. Moving the avatar in utterly normal ways, which meant distorted
movements on the part of the live performers. This was fascinating since
it resonated back to the performers, who themselves were twisted in their
movement. It was amazing choreography, created to 'normalize' the
equipment output.
Foofwa and returning the avatar movements to 'real' life. [examples,
explanation.] The 'smearing' of divides between real and virtual, each
borrowing from, and resonating with, the other.
[Examples]
Note that with all of this, there are no programming errors, only other
avenues, glitches, to be explored. So the aesthetics and phenomenology of
glitch are important here as well. In virtual worlds and with motion
capture, there are in particular 'edge' glitches - within and without
gamespace boundaries - that define, in a sense, _all_ the possibilities of
the avatar, _all_ the possibilities of escape and normalcy...
The imaginaries I work with - virtual worlds; 3d modeling; 3d printing;
very low frequency (VLF) radio; scanner and shortwave radio; augmented
reality; playing music; codework (an entangled amalgam of code, writing,
and computer 'debris'; even birding, which requires abstractions ranging
from migration routes to morphs.
Finally, the idea that the virtual has always been with us, that the body
is always already inscribed, that culture goes all the way down, that
inscription and the digital are entangled amalgams as well, and that
abjection underlies everything, as well as pain, suffering, and death, all
part of it.
Thank you -
=======================================================================
Dance description (for the empyre email list, highly edited here)
I've been following this discussion and thought the best way I might
participate is to describe the work that I've done with Foofwa d'Imobilite
and others over the past decade or so. We went from using video and audio
tracks accompanying choreography, to work in Blender and Poser. The Poser
work was created from bvh (Bio-Vision Hierarchy) files produced with
motion capture (mocap) equipment that used 21 sensors electromagnetically
interacting with an antenna. The antenna fed sensor signals into a hard-
wired 486 microprocessor that output coordinates; these were fed into a
second computer that created the bvh files themselves. we modified the
sensors in a number of ways - some through the software interface, and
some with limb assignment and position. We did a piece called heap for
example - the sensors were dropped in a heap and the bvh file fed into
Poser. We did a star piece, arranging the sensors in a star formation on
the floor and inverting it by exchanging +r from a sensor position to -r.
We also reassigned sensors in several ways - dividing them between two
bodies, remapping inversely onto a single body, and so forth. All of this
produced bvh/Poser mannequins that were used as projections in live per-
formance, or chroma-keyed over dance/performance video.
All of this work was at West Virginia University's Virtual Environments
Lab, headed by Frances van Scoy. I received an NSF consultancy through
Sandy Baldwin and NYSCA grant; through the former, I had a grad assistant
from software engineering, Gary Manes, to assist me. We went into the
mocap software itself and Gary rewrote it, creating a dynamic/behavioral
filter interface, which would produce transforms from the sensor output -
before the 3-d assignment to bvh was made. This was modeled on graphic
software filtering, but the assignments were different - we applied a
function f(x) to the coordinates and/or modified the coordinate mechanism
or input streams themselves. The bvh files that were produced were sent
into Poser for editing; in some cases, Poser mannequin video was output.
But more and more, we edited in Poser to format the bvh for upload to
Second Life; this way we had live 3-d performance based on the transforms.
This performance could interact within Second Life itself - with other
online performers and audience - or through projection, without Second
Life, in real-space where performers might interact with the avatars.
The bvh files are complex and avatars perform, most often at high-speed,
with sudden jumps and motions that involve them intersecting with them-
selves. The motions appeared convulsive and sometimes sexualized. Foofwa
d'Imobilite used projections direct from Poser - about 100 files - as part
of Incidences, a piece produced in Geneva and widely shown. Foofwa, along
with Maud Liardon and my partner, Azure Carter, also imitated avatar move-
ment - and this fed back, from dance/performance into programming and pro-
cessing; at times it has been impossible to tell whether a particular
motion stream originated on- or off-line.
In SL, everything is pure, digital, protocol, numeric; by 'smearing' the
animation input, avatar appearance, and location, we create in-world and
out-world experiences that stray from body and tend towards choratic and
pre-linguistic drives. We've performed a lot at various limits of SL - on
sim edges for example, or at 4k 'up', where the physics changes. The
output is the usual - audience in-world or out-world, as well as video and
stills.
Foofwa, Maud, Azure, and myself all traveled to the Alps where avatar work
was re-enacted live; the performances were on the edge of the Aletsch
glacier. (This was sponsored by a Swiss grant.) What was interesting most
to me here was the development and performance of a field - Foofwa dancing
with a VLF (very low frequency) radio antenna, for example - his body
coupled and modified the electromagnetic capacitance surrounding the wire.
We had done this indoors with Foofwa and Azure; outdoors, against the
glacier, spherics formed a deep part of the content. This also paralleled
work we did with the mocap sensors at WVU - using high-strength magnetics,
we modified the local fieldlines, almost as if we were modeling general
relativity's 4-space gravity/mass interaction - the results were similar.
I'm fascinated by these 'cosmologies in the small'; at the same time, want
to avoid any easy and false metaphoric equivalence with scientific theory.
As for the theory of the work we're doing, at least from a phenomenologi-
cal viewpoint, I've put up http://www.alansondheim.org/sltheory.txt which
has also been published as a book.
