Changing LiDAR backend to ARKit
Initial testing of AVFoundation point clouds was promising, as we were able to draw maps like this:
However, we found that the point clouds provided by AVFoundation were only spatially accurate under certain conditions. Specifically, if the phone was too close or too far, the quality drops. Below is an example of a spatially inaccurate point cloud when we point the camera too close; this is a top-down view of a 90 degree wall, but it appears to be quite a bit more than 90 degrees.
This error also seems to be quite unpredictable, as other cases exhibit other errors. For example, here the point cloud is warped along the y-axis instead.
We found that the simple switch from AVFoundation to ARKit mostly solved these issues. For example, here is that 90 degree wall again:
It does appear that ARKit is a bit slower than AVFoundation, and will require some optimization on our part to get point clouds to stream faster. We are also considering switching off the rerun.io visualizer and instead building our own, as the rerun interface has been quite buggy.s