Piscine Virtual Reality

[Image: From “Putting the Fish in the Fish Tank: Immersive VR for Animal Behavior Experiments” by Sachit Butail, Amanda Chicoli, and Derek A. Paley].

I’ve had this story bookmarked for the past four years, and a tweet this morning finally gave me an excuse to write about it.

Back in 2012, we read, researchers at Harvard University found a way to fool a paralyzed fish into thinking it was navigating a virtual spatial environment. They then studied its brain during this trip that went nowhere—this virtual, unmoving navigation—in order to understand the “neuronal dynamics” of spatial perception.

As Noah Gray wrote at the time, deliberately highlighting the study’s unnerving surreality, “Paralyzed fish navigates virtual environment while we watch its brain.” Gray then compared it to The Matrix.

The paper itself explains that, when “paralyzed animals interact fictively with a virtual environment,” it results in what are called “fictive swims.”

To study motor adaptation, we used a closed-loop paradigm and simulated a one-dimensional environment in which the fish is swept backwards by a virtual water flow, a motion that the fish was able to compensate for by swimming forwards, as in the optomotor response. In the fictive virtual-reality setup, this corresponds to a whole-field visual stimulus that is moving forwards but that can be momentarily accelerated backwards by a fictive swim of the fish, so that the fish can stabilize its virtual location over time. Remarkably, paralyzed larval zebrafish behaved readily in this closed-loop paradigm, showing similar behavior to freely swimming fish that are exposed to whole-field motion, and were not noticeably compromised by the absence of vestibular, proprioceptive and somatosensory feedback that accompanies unrestrained swimming.

Imagine being that fish; imagine realizing that the spatial environment you think you’re moving through is actually some sort of induced landscape put there purely for the sake of studying your neural reaction to it.

Ten years from now, experimental architecture-induction labs pop up at universities around the world, where people sit, strapped into odd-looking chairs, appearing to be asleep. They are navigating labyrinths, a scientist whispers to you, trying not to disturb them. You look around the room and see books full of mazes spread across a table, six-foot-tall full-color holograms of the human brain, and dozens of HD computer screens flashing with graphs of neural stimulation. They are walking through invisible buildings, she says.

[Image: From “Putting the Fish in the Fish Tank: Immersive VR for Animal Behavior Experiments” by Sachit Butail, Amanda Chicoli, and Derek A. Paley].

In any case, the fish-in-virtual-reality setup was apparently something of a trend in 2012, because there was also a paper published that year called “Putting the Fish in the Fish Tank: Immersive VR for Animal Behavior Experiments,” this time by researchers at the University of Maryland. Their goal was to “startle” fish using virtual reality:

We describe a virtual-reality framework for investigating startle-response behavior in fish. Using real-time three dimensional tracking, we generate looming stimuli at a specific location on a computer screen, such that the shape and size of the looming stimuli change according to the fish’s perspective and location in the tank.

As they point out, virtual reality can be a fantastic tool for studying spatial perception. VR, they write, “provides a novel opportunity for high-output biological data collection and allows for the manipulation of sensory feedback. Virtual reality paradigms have been harnessed as an experimental tool to study spatial navigation and memory in rats, flight control in flies and balance studies in humans.”

But why stop at fish? Why stop at fish tanks? Why not whole virtual landscapes and ecosystems?

Imagine a lost bear running around a forest somewhere, slipping between birch trees and wildflowers, the sun a blinding light that stabs down through branches in disorienting flares. There are jagged rocks and dew-covered moss everywhere. But it’s not a forest. The bear looks around. There are no other animals, and there haven’t been for days. Perhaps not for years. It can’t remember. It can’t remember how it got there. It can’t remember where to go.

It’s actually stuck in a kind of ursine Truman Show: an induced forest of virtual spatial stimuli. And the bear isn’t running at all; it’s strapped down inside an MRI machine in Baltimore. Its brain is being watched—as its brain watches the well-rendered polygons of these artificial rocks and trees.

(Fish tank story spotted via Clive Thompson. Vaguely related: The Subterranean Machine Dreams of a Paralyzed Youth in Los Angeles).

Optical Spelunking

[Image: The CAVE at the Desert Research Institute in Reno, now called the CAVCaM].

I mentioned a week or two ago that I had been out to Reno, Nevada, visiting, among other things, the Desert Research Institute, where Nicola Twilley of Edible Geography, Mark Smout of Smout Allen, and I began a roadtrip down to Los Angeles, through San Francisco—less a city than a peninsular amphitheater of conflicting microclimates—by way of the Virtual Reality CAVE that you see pictured here.

[Image: Daniel Coming, Principle Investigator of the CAVCaM, manipulates geometries that don’t exist, and we photograph him as he does so].

The facility is no longer called the CAVE, I should add; it’s now the CAVCaM, or Center for Advanced Visualization, Computation and Modeling. CAVCaM “strives to maintain a state-of-the-art visualization system, improve data collections, simulations, and analyses of scientific information from the environment.”

Advancements will create new capabilities for multidisciplinary research, produce top tier visualization environments for use by the broader scientific community, and offer opportunities to improve management decisions including prediction, planning, mitigation, and public education throughout Nevada and the world.

It also blows the minds of landscape theorists and practitioners in the process.

[Image: Touring virtual light].

In most of the photos here you see Matthew Coolidge from the Center for Land Use Interpretation, Bill Gilbert from the Land Arts of the American West program, and activist landscape historian and theorist Lucy Lippard all trying their hands at setting virtual forest fires, chasing digital terrains off cliffs, and navigating a world of overlapping proximities that sewed together around us like high-end neurological garmentry—a perfectly tailored world of pharaonic nonexistence, standing in tombs of imagery and light—to become almost seamlessly 3D. Glimpsing, in advance, possible afterlives of the optic nerve.

[Image: Cthulhoid satellites appear in space before you, rotating three-dimensionally in silence].

Of course, these photos also show the inteprid Dr. Daniel Coming, “Principle Investigator” of the CAVCaM—a fantastic job title, implying that this strange machinic environment that the DRI has built isn’t so much put to use, in a dry, straight-forward, functional way, but investigated, researched, explored. Daniel showed us all how to use the hand controls, putting on a display of virtual light and shadows. Objects that were never built, reflecting light that isn’t real.

We were all there on an invitation from the staff of the Nevada Museum of Art—who don’t appear in these photographs, but were absolutely key in making this tour happen.

[Images: Photos by BLDGBLOG and Nicola Twilley].

For whatever reason, meanwhile, that last photograph, above, featuring Matthew Coolidge, Bill Gilbert, and Lucy Lippard seemingly entranced—as we all were—by this new altarpiece of virtual surfaces, reminds me of the final lines from R.S. Thomas’s old poem “Once”:

Confederates of the natural day,
We went forth to meet the Machine.

Or perhaps it was the Machine that has come to meet us.

[Image: The CAVCaM reboots after a universe of simulation].