Mons Niklas Schak Composer & Sound Artist
Sound Perspectives in Hyperrealistic Spaces

· Research diary

AR Prototype — Spatial Audio in Your Pocket?

One of the central questions in this research project is how spatial sound behaves differently across playback formats — from a physical forest installation to a VR headset to a phone held in your hand. Today I ran a small experiment to find out what happens when you strip the experience down to its most portable form: augmented reality on an iPhone.

I took an existing photogrammetry scan of one of the Cars Choir cars (made with Polycam (opens in a new tab) during the summer sessions at Kyrkö Mosse) and built it into a USDZ file — the format Apple uses for AR Quick Look. The model needed significant rescaling (Polycam’s output was 39 times too large), but once calibrated to the real car’s dimensions — 3.5 metres long, 1.30 metres high — it placed convincingly on the floor in AR.

The interesting part was the sound. I prepared two versions of the same file, each with a 24-second audio loop embedded directly in the 3D model:

  • Version A: A binaural recording, captured with a binaural microphone placed inside the car at Kyrkö Mosse. The sound is the actual installation playing through the car in its original location — the spatial impression is baked into the recording itself. Played as ambient (non-spatial) audio in the AR model.
  • Version B: A mono recording of the same moment, captured with a Zoom MS microphone. Same source — the installation playing through the car on location — but recorded as a single channel, then played as spatial audio positioned at the car, with the phone calculating direction and distance in real time.

The result? Almost no perceptible difference. Walking around the car in AR, the spatial version barely shifted. The reason is clear: AR Quick Look uses a simplified spatial audio engine — basic distance attenuation and stereo panning, but no HRTF processing and no head-tracking. The phone knows where you are relative to the car, but not which way your ears are pointing.

Try It Yourself

If you have an iPhone or iPad, you can download the two models and experience the experiment first-hand. Tap a file to place the car in your room — make sure your sound is on and try walking around it.

↓ Download Version A — binaural (non-spatial)

↓ Download Version B — mono (spatial)

Requires iOS 12 or later. Open the file on your device — it will launch AR Quick Look automatically.

This is a useful finding for the research. It confirms that the spatial audio experience I’m working towards — where the listener’s movement through space fundamentally shapes what they hear — cannot be meaningfully prototyped on a phone. The phone is fine for testing the visual model and auditioning which sounds work with which cars. But the actual spatial dimension requires a VR headset with full six-degree-of-freedom tracking, which is the next step: Unreal Engine with Steam Audio on Meta Quest 3.

In other words, the “hyperrealistic space” in the project’s title needs a format that lets you be inside the space — not just looking at it through a window.

From the research diary for Sound Perspectives in Hyperrealistic Spaces. All diary entries →

Sound Perspectives in Hyperrealistic Spaces