Landmark Calibration Sample for ARKit

This sample project demonstrates how to use ARKit together with posemesh domains. Portals are special QR codes physically placed on the floor, placed into a persistent coordinate system. By scanning any portal, the phone finds its position within domain coordinates.

The device tracks its own movement using ARKit, and whenever a portal is detected it aligns ARKit's coordinate system to the physical world.

More specifically, this sample project uses the Posemesh SDK for iOS to:

• Use the device's camera feed to scan for:
  • An ArUco marker with the number 17 (ArUco original format, 6x6 bits, minimum hamming distance between any two codes = 3, 1024 codes).
  • Auki portals placed with the DMT app.
• Align the ARKit world coordinate system (calibration) to domain coordinates, based on detected markers/portals.

How to run

1. Download the Posemesh SDK for iOS ARM64 device build or clone the Posemesh SDK repository and build it yourself.
2. In the Xcode project add the Posemesh SDK iOS framework to the project (and include its path in Framework Search Paths if needed).
3. To start building on the posemesh, jump ahead to the "Full Demo" section below. Optionally, for a quick first test, follow the "Quick Demo" with a single ArUco marker.

Quick Demo: Single ArUco marker

If you're printing it, make sure that either (a) no scaling is applied, or (b) you measure the printed marker and set the halfMarkerSideLength in PosemeshSample.swift to half that size in meters (e.g. a 10 cm side means halfMarkerSideLength = 0.05).

The marker is assumed to be placed horizontally. If you want it to be vertical (e.g., displayed on a screen), in hardcodedLandmarks() in PosemeshSample.swift change the landmark coordinates from Z to Y and negate them:

// for all lp* points change from
// lp*.z = value
// to
// lp*.y = -value
// e.g.
lp0.z = -halfMarkerSideLength
// becomes
lp0.y = halfMarkerSideLength

Full Demo: Landmark Calibration

1. Sign up as a developer at https://console.auki.network/.
2. Create a domain.
3. Generate and print a few portals as described in Domain Setup: Portals.
4. Put portals on your floor and use the Domain Management Tool (DMT) app to place them into your domain, as described in Domain Setup: DMT.
5. Use Web DMT to generate landmarks for your portals. This step is currently manual, but will be automated soon.
   1. Using your developer credentials from step 1, log into https://dmt.auki.network.
   2. Select your domain.
   3. Go to the Portals page, and make sure your portals are shown.
   4. In the context menu above the list, click "Apply as Landmarks".

You now have the data for all portals in the format needed for Landmark Calibration, stored as domain data. Tip: You can also view domain data in the Auki console.

6. Open LandmarkCalibrationSampleARKit/Posemesh/AukiDomainDataSource.swift and set the aukiUsername, aukiPassword and aukiDomainID to allow the app to download the landmarks domain data.

Now you can build and run the app (if not, see 'Troubleshooting' below). The landmarks for all your portals from step 5 are downloaded at the start using the credentials you provide. Scan any of your portals to let the app calibrate into domain coordinates.

For any questions or ideas, feel free to reach out on our Discord or email.

Troubleshooting

App fails to compile, with errors about Posemesh framework or Posemesh types missing.

Fix:

Make sure you've added the Posemesh.framework correctly to the project. Whether you've compiled from source or downloaded a pre-compiled version, ensure xcode finds the file. The easiest way is when you add the framework to choose "Copy files to destination".

Also make sure you've added the Posemesh.framework under the settings for the Target "LandmarkCalibrationSampleARKit" > "Build Phases" > "Link Binary With Libraries". This is essential for the sample to use the compiled framework.

Swift compiler version mismatch

/Users/robin/Documents/GitHub/LandmarkCalibrationSampleARKit/Posemesh.framework/Modules/Posemesh.swiftmodule/arm64-apple-ios.private.swiftinterface:1:1: error: failed to build module 'Posemesh'; this SDK is not supported by the compiler (the SDK is built with 'Apple Swift version 6.0 effective-4.1.50 (swiftlang-6.0.0.9.10 clang-1600.0.26.2)', while this compiler is 'Apple Swift version 6.1.2 effective-4.1.50 (swiftlang-6.1.2.1.2 clang-1700.0.13.5)'). Please select a toolchain which matches the SDK.
// swift-interface-format-version: 1.0

If you see errors similar to above, it may be that your xcode version is different than what the framework was compiled with.

Fix:

If you downloaded the framework from the posemesh github, please file an issue telling which file you donwloaded and your xcode version.

Alternatively, if you compile the posemesh from source, make sure you compile the framework and the sample repo using the same xcode version.

App compiles but crashes on launch, with errors about the Posemesh framework, e.g.:

Library not loaded: @rpath/Posemesh.framework/Posemesh
  Referenced from: <C5C6E663-C2AA-3F60-AC7E-DE098673CA4F> /private/var/containers/Bundle/Application/131C10FF-6F58-473B-92FE-AC98C9F39784/LandmarkCalibrationSampleARKit.app/LandmarkCalibrationSampleARKit.debug.dylib
  Reason: tried: '/private/var/containers/Bundle/Application/131C10FF-6F58-473B-92FE-AC98C9F39784/LandmarkCalibrationSampleARKit.app/Frameworks/Posemesh.framework/Posemesh' (no such file),
  ...

Fix:

Make sure the compiled Posemesh.framework is set to be embedded in the app. Click your xcodeproj (root of the left menu) in Xcode, go to General tab, and check under "Frameworks, Libraries and Embedded Content". You should have the Posemesh.framework there, and set it to "Embed & Sign".