facebook surround 360 redefines DIY panoramic imagery

facebook surround 360 is officially open-sourced: the high-quality 3D-360 hardware and software video capture system includes the hardware, camera design and software stitching technology to make end-to-end 3D-360 video capture possible in one system — from shooting to video processing — for the homemade immersive multimedia experience.

both the hardware design and software play a role in achieving a high-quality output. facebook have previously written about the hardware design of the camera and its importance in decreasing the processing time for the footage. even though high-quality optics and machine-fabricated rigs are used, there is still enough variation between camera lenses that without software corrections, results are not viewable in stereo (you see double vision and not 3D). the stitching software takes the images captured by the 17 cameras in surround 360 and transforms them into a stereoscopic 360 panorama suitable for viewing in VR. this software vastly reduces the typical 3D-360 processing time while maintaining the 8k-per-eye quality which is optimal for the best VR viewing experience.

the development team tried various camera/software combinations while developing prototypes for a 3D-360 camera. the first attempt was four pairs of cameras pointing out in a square, spaced roughly eye-distance apart. then they tried to stitch the left and right eye images into a seamless panorama with alpha blending, which is a basic technique for mixing images together to create a smooth transition. it looked great when pointing straight out in the direction of the real cameras, but the stereo was off at the corners. another issue with this approach is that as you rotate around, your eyes have to point in different directions to look at objects a constant distance away, which causes eyestrain.

there are more sophisticated ways to construct stereo panoramas than basic alpha blending and stitching. one approach is to take slices of the images captured by a camera that rotates around a fixed axis. this approach produces panoramas that have proper vergence and creates more consistent perception of depth than basic stitching. however, it cannot be applied to scenes where anything is moving. going one step further, optical flow is used to simulate a rotating camera, by ‘view interpolation’ from a small number of cameras arranged in a ring; this approach can capture scenes with motion. the final product is a pleasant visual experience which can be viewed below.