[Image: “Two images of the same room, one reconstructed from video footage of a bag of chips within the room (top) and the other photographed directly (bottom),” as described by Scientific American. Images courtesy Jeong Joon Park.]
“Researchers have now found that by filming a brief video clip of a shiny item, they can use the light flashing off it to construct a rough picture of the room around it,” Scientific American reports. “The results are surprisingly accurate, whether the reflections come from a bowl, a cylinder or a crinkly bag of potato chips.”
It comes down to mathematically modeling “what a known object will look like—how light will reflect off it—when it is placed in new surroundings,” such that you can then reconstruct the proper orientation of what it reflects.
There’s a lot more in the original article, but what immediately struck me about this was how this technology could be used for crime or espionage, both.
You send an unsuspecting group of school kids into a target building, carrying highly reflective silver balloons, or you wear a slyly reflective and precisely designed item of clothing into a business meeting: in both cases, a photographer on a roof across the street or hidden in a park nearby snaps away through a telephoto lens. The reflections spilling off in all directions are like a 360º spherical photograph of the building interior—the art on the walls, the position of furniture. The location of a safe.
Think of the Japanese pop star who was tracked by a stalker after he deduced her location by analyzing the reflection in her eye in a selfie. Every mirrored surface becomes a security leak—“Las Meninas” as burglary tool.
[Image: “Las Meninas” (1656) by Diego Velázquez; if my reference to this painting makes absolutely no sense in the present context, it’s because I’m being pretentious and indirectly referring to Michel Foucault’s The Order of Things, where he discusses the painting’s use of internal reflection.]
Of course, you may also recall that sounds can be reconstructed from the vibrations of distant objects: “Researchers at MIT, Microsoft, and Adobe have developed an algorithm that can reconstruct an audio signal by analyzing minute vibrations of objects depicted in video. In one set of experiments, they were able to recover intelligible speech from the vibrations of a potato-chip bag photographed from 15 feet away through soundproof glass… In other experiments, they extracted useful audio signals from videos of aluminum foil, the surface of a glass of water, and even the leaves of a potted plant.”
It’s worth noting here how potato chip bags pop up in each example. Ocean’s 14 will open with a surreptitious potato chip delivery…
In any case, political dissidents, high-value corporate CEOs, and adversarial diplomatic attachés will never be safe again. Just a brief reflection from a cigarette lighter or a piece of silverware, just a tiny ripple of sound across the leaves of an exotic orchid in the center of a dinner table, and someone across the city with a telescope has your bank passcode, the location of your home safe, and a complete 3D map of your building interior, even down to where your security guards are sitting.
[This is only somewhat related, but recall that an engineer at Carnegie Mellon has developed “a long-range iris scanner that can identify someone as they glance at their rear-view mirror” in a moving vehicle, Rob Meyer reported for The Atlantic back in 2015.]
Hah. So wearing that tinfoil hat becomes a self-fulfilling prophecy.
Or it was an illicit interior-mapping tool all along…
A vast government conspiracy to generate fake government conspiracies, building an army of tinfoil hat wearers convinced they’re anti-government but who are, in fact, working for the government. And of course, the conspiracy theorists who uncover the operation and are suddenly forced to confront their own kind and try to convince them of the real “truth”. Which is in itself just a way to distract them from the real conspiracy… and now my brain hurts.
On an unrelated note, after reading this article I’m going to start referring to the matte side of aluminum foil as “incognito mode”.
If that was a comic book, I’d read it… Like Looking Glass in Watchmen refigured as informant, conspiracist, Deep State activist, and mirror-truther, all in one.
the next level is then extracting the sound from the same bag of chips. so you’ll have a future Gene Hackman from The Conversation in the same room with a future David Hemmings from Blowup working together to get the full 4D experience.
Derek Muller has a beautiful video on his channel about extracting sound from an image:
I literally just realized that you talk about this exact same thing in the article. my bad.
No apologies necessary—every comment of yours is a pleasure. A Marvel Cinematic Universe-like crossover event—bringing together The Conversation and Blowup—would be quite an experience, indeed.