Snap Inc shook things loose at the Augmented World Expo in Long Beach, California, by giving people a closer look at some of the game-changing advancements being done on augmented reality (AR) technology. This ranges from a real-time, on-device image diffusion model for colorful AR to new generative AI tools, specifically custom-made for the AR creator. This move will make a substantial increase in the integration of generative AI within AR and probably is going to revolutionize how users will experience and interact with their surroundings through their smartphones.
According to Snap co-founder and CTO Bobby Murphy, two of the most important qualities in generative AI for AR are speed and efficiency. Traditional AI image diffusion models impress a lot, but there is substantial acceleration needed to really make them impactful for real-time applications. The company’s latest innovation solves this by developing a small enough model that can even run on a smartphone, fast enough to re-render frames in real time at the guidance of text prompts.
“We’re introducing new augmented reality experiences using GenAI for Snapchatters and our AR developer community,” Murphy said, touting the potential of this tech to change how people experience AR.
GenAI comes to enable innovative kinds of Lenses that can improve user experiences.
Snapchat users can see Lenses that bring this real-time generative model into action. The new lenses will help users create vivid augmented reality transformations in an instant. For example, while earlier AR features could support only basic effects like putting a hat on a person’s head, the new AI-powered lenses will offer fluid interactions like the natural movement of the hat with the movement of the person’s head or adjusting to various lighting conditions in real time.
Empowering Creators with Lens Studio 5.0
Along with these consumer-facing innovations, Snap announced Lens Studio 5.0, a suite of generative AI tools for AR creators. These tools reduce the overall time to build high-quality Lenses and give faster production. Key features include the following:
- AI Assistant: Inside Lens Studio, simply type a question to the AI Assistant, and you’ll get a helpful response quickly, so you can get unstuck and keep creating.
- ML Face Effects: Create the next trending selfie experience with a custom ML Face Effect. You can generate a highly realistic ML model that’s unique every time.
- Immersive ML: Generate custom stylization effects that apply a realistic transformation over the user’s face, body, and surroundings in real time.
- 3D Asset: With a simple text or image prompt, generate a 3D asset of your choice in minutes.
- Head Morph: Unlock character creation — like space aliens, wizards, monsters, and more — with a text or image prompt that generates 3D character heads using our Face Mesh technology.
- Face Mask, Texture, and Material Generation: Generate your own materials and textures faster and easier than ever or create a face mask in minutes for a transformative selfie experience.
In addition, this latest version, Lens Studio 5.0, is equipped with an AI assistant that will be alongside creators to answer any questions for further smoothing of the development process.
It’s not just technology, Snap is making a lot more innovative use of AR. The company teamed up with London’s National Portrait Gallery to create Lenses inspired by iconic portrait styles. That means leading a collection of Snapchatters who opt for any portrait-style Lens, take a Snap, and then submit it to the museum’s “Living Portrait” projection wall. This has been an initiative within Snap’s broader effort to retain and grow users with singular, cutting-edge AR experiences.
With further evolution in the AR and VR industries, Snap’s foray into real-time generative AI raises the bar. Integrating generative AI inside AR applications is a harbinger of times when AR and VR experiences will not be only more dynamic and interactive but also more accessible to creators and users alike.
While other competitors like Meta and Apple are also focused on AI-driven AR technologies, Snap’s lead in this department gives them a leg up on the competition. Continued development of AR glasses and a possible later foray into mixed reality VR headsets put Snap at the forefront of taking an important role in the future of AR and VR.
