About
Services
Our Work
Observations and innovations from WWDC 2024
Dawson Metzger-Fleetwood
When Apple invites your team to join them at WWDC as they roll out their latest platform updates, you know you’re doing something right. About Objects was thrilled to experience features and functionality that will impact developers and users on a daily basis.
VisionOS
Along with visionOS 2, Apple is releasing a slew of forward-thinking APIs and developer tools.
- New Volumetric APIs will allow fully immersive spatial experiences to run alongside other apps. They will also give you greater control over how objects resize when placed in space and allow ornaments to be affixed to volumes.
- Room Anchors and Object Tracking APIs will improve scene understanding, allowing you to respond to the way users move between rooms. Add to that the ability to attach virtual content to physical objects.
- TabletopKit will make it easier to develop shared experiences centered on tables.
- RealityKit 4 will give you greater control over your 3D scene. This includes APIs for adding materials, creating shader-based hover effects along with low-level control over mesh and textures using Metal compute shaders.
- Xcode will give you the ability to inspect 3D scene content while view debugging.
Our team was particularly excited about the latest enterprise APIs, which will offer new access to the main camera, the Neural Engine, and more
Apple Intelligence
Apple Intelligence won’t just benefit consumers. With App Intents and App Entities, Siri can take actions in your app while surfacing information from your app at appropriate times. Siri’s ability to utilize your app while executing complex user workflows will only grow in importance as AI becomes more capable.
Machine Learning
Updates to Core ML will make it easier to optimize and run generative models—such as transformer-based language models and diffusion-based image models—on device. From composable weight compression to more efficient state management, Apple silicon is home to some of the most powerful and widespread edge computing ever performed. Frameworks like Translation, Vision, and Natural Language will allow developers to easily embed translation, pose detection, and multilingual contextual embedding features into their apps.
One personal highlight occurred during a series of unrecorded sessions where Apple laid out design best practices for apps that make use of Generative AI. For example, we learned how to effectively mitigate the controllability risks posed by language and image models, while keeping the user fully engaged during inference.
Xcode and Swift
Xcode 16 also had its share of major announcements. Among my favorites were predictive code completion and their latest testing APIs.
- Powered by a model trained specially for Swift and Apple SDKs, predictive code completion allows developers to spend less time typing and more time innovating and problem-solving. Plus, unlike Github Copilot and other similar tools, your code never leaves your computer. Because the model is trained on Apple SDKs, developers will no longer need to parse through out-of-date or hallucinated information produced by other LLMs.
- New testing APIs promise to streamline writing and reading tests. Results will appear in-line with your code, making debugging easier. New tags, macros, and parametrization will let developers selectively run tests, capture complex expressions, generate detailed output, and run the same test across multiple values.
- Swift also introduced data race safety. Before now, protecting applications from data races has been entirely on the programmer’s shoulders. This is incredibly difficult to do correctly and keep correct as applications grow in complexity and size. Worst of all, data races are unusual and unpredictable, making them difficult to reliably test for. . Swift 6 eliminates the problem entirely by detecting potential data races at compile time. No matter the size or complexity of the app, accidental data races are no longer an issue.
What I’ve shared makes up only a portion of the announcements Apple made at the conference. There were updates to macOS, iOS, iPadOS, and watchOS, which I can post about later. Looking back on the experience, what stood out most about WWDC was how widespread and welcoming the Apple developer community is. I for one cannot wait to see what other talented programmers do with these tools at their fingertips.
