Jetpack XR beta: what Android teams must change to ship XR a

Your ARCore-based app runs at 60fps on a phone, and now your client wants it on a headset with spatial windows and passthrough cameras. The Jetpack XR SDK core libraries just hit beta, and the migration path isn't just swapping a dependency. It's rethinking how your app handles rendering, input, and especially on-device inference.

Google announced the Jetpack XR SDK core libraries beta earlier this month, bringing a unified API surface for spatial computing across compatible Android XR devices (Android Developers Blog). The beta matters because it standardizes what was previously fragmented across vendor SDKs. But if your team has been building with ARCore and Sceneform (or its successors), the architectural shift is real.

What the beta actually changes

The Jetpack XR SDK core libraries introduce a spatial app model that differs from the phone-first Android lifecycle most teams know. Your activity doesn't just resume and pause. It enters spatial mode, manages multiple spatial windows, and handles input from controllers, hands, and eyes simultaneously.

If you've been building AR scenes with ARCore sessions, the SDK's spatial foundation layer replaces some of that plumbing. You get a managed spatial activity, compositor hooks for rendering to XR surfaces, and a tracking API that abstracts headset pose data. The trade-off: you give up fine-grained ARCore session control for a higher-level model that handles lifecycle and rendering surface management for you.

For teams running on-device ML, this matters immediately. Your inference pipeline needs to coexist with the XR render loop. On a phone, you can afford to block the camera frame callback for 30ms while your model runs (Android Developers Blog). On a headset, that same stall drops frames and induces motion sickness. The SDK's frame scheduling model assumes you're rendering at the display's native refresh rate, and inference has to fit in the gaps.

Where on-device AI fits in the XR pipeline

We've been building on-device AI systems for hospitality and retail clients, and the XR integration pattern is the same mistake teams keep making: treating ML inference as a background task that doesn't interact with the render loop.

In a phone app, your pipeline looks like: camera frame arrives, you run inference, you update the UI. The latency budget is forgiving because the user isn't wearing the display on their face.

In XR, the pipeline is different. The compositor expects a frame every VSync. Your on-device model needs to produce results that feed into the spatial scene graph without blocking the render thread. This means you need a producer-consumer pattern where inference runs on a dedicated thread or coroutine, publishes results to a shared state object, and the render thread reads that state without locking.

For models like Qwen3-8B or Phi-3 running on-device, the inference time per token matters less than how you schedule it relative to the frame deadline. If you're running a vision model for object detection in a spatial shopping app, you need to pipeline the inference so that frame N+1 uses the results from frame N, not the current frame. This adds one frame of latency but keeps the render loop stable.

We've found that on-device model selection for XR comes down to thermal budget, not just latency. A headset running a vision model at 30Hz will throttle within minutes if the model is too large. You need a model that fits the thermal envelope of a battery-powered, head-mounted device. This is where lighter models like Phi-3 or quantized variants of Qwen3 earn their place. If you need help selecting or optimizing a model for XR thermal constraints, our custom models service handles this directly.

Migrating from ARCore: what to keep, what to throw away

If you have an existing AR app, here's what changes when you adopt the Jetpack XR SDK core libraries.

Keep your ML inference code. Your TFLite or PyTorch Mobile models, your preprocessing and postprocessing pipelines, your quantization strategy. None of that changes. The SDK doesn't touch your ML stack.

Throw away your ARCore session management code. The SDK's spatial activity handles session lifecycle, tracking, and surface creation. Your manual Session.resume() and Session.pause() calls go away. This is the biggest code deletion in the migration and also the most error-prone, because your app likely has implicit dependencies on session state that you didn't realize existed.

Rewrite your rendering layer. If you were using Sceneform or OpenGL directly, you need to move to the SDK's rendering surface model. The SDK provides compositor integration that handles distortion correction and layer composition, but your draw calls need to target the spatial surface, not a standard SurfaceView.

Restructure your input handling. Phone AR apps assume touch input. XR apps need to handle controller raycasts, hand tracking gestures, and eye tracking as first-class input sources. The SDK provides a unified input API, but your app logic needs to decide what each input source controls.

The thermal question nobody asks until production

The biggest risk we see in XR app development isn't rendering or input. It's thermal throttling. Headsets have aggressive thermal limits because they're strapped to someone's face. If your on-device model pushes the SoC too hard, the system will throttle your app, and you'll see frame drops that have nothing to do with your render loop.

You need to profile your model's power consumption on actual XR hardware, not just a dev kit. Measure sustained inference over 10 minutes, not just peak performance. If your model causes throttling, you need a smaller model, a lower inference frequency, or a hybrid approach where heavy inference runs on a companion device.

For teams exploring this, our on-device AI work includes thermal profiling for XR targets. If you're trying to estimate compute costs for a hybrid edge-cloud architecture, our AI cost estimator gives you a baseline. And if you need a sounding board for your XR migration architecture before you commit engineering time, our AI consulting team can pressure-test your plan.

We ran into this exact problem building RunHotel, our on-device voice AI for hotels. Voice inference on a headset has to share the SoC with spatial rendering, and the thermal ceiling is lower than any phone we've targeted. The lesson was simple: profile early, profile on real hardware, and pick your model based on sustained thermal output, not benchmark latency.

What to do this week

Pull the Jetpack XR SDK core libraries beta into a throwaway branch, port your simplest ARCore activity to the spatial activity model, and measure the frame timing with your existing on-device model running alongside it. That single experiment will tell you whether your current ML pipeline survives the XR transition or needs a rewrite before you commit to the full migration.