- USD is a high-performance extensible software platform for collaboratively constructing animated 3D scenes, designed to meet the needs of large-scale film and visual effects production.
- USD enables robust interchange between digital content creation tools with its expanding set of schemas, covering domains like geometry, shading, lighting, and physics.
- USD’s unique composition ability provides rich and varied ways to combine assets into larger assemblies, enables collaborative workflows so that many creators can work together with ease, and more.
[https://openusd.org/release/index.html]
Universal Scene Description (USD) is a framework for interchange of 3D computer graphics data. The framework focuses on collaboration, non-destructive editing, and enabling multiple views and opinions about graphics data. USD is used in many industries including visual effects, architecture, design, robotics and CAD. It is developed by Pixar and was first published as open source software in 2016, under a modified Apache license. Pixar, Adobe, Apple, Autodesk and Nvidia, together with the Joint Development Foundation (JDF) of the Linux Foundation, announced the Alliance for OpenUSD (AOUSD) on August 1, 2023 to “promote the standardization, development, evolution, and growth of Pixar’s Universal Scene Description technology”.
File formats
File formats used by the specification include:
- .usd, which can be either ASCII or binary-encoded
- .usda, ASCII encoded
- .usdc, binary encoded
- .usdz, a package file which is a zero-compression, unencrypted zip archive, which may contain usd, usda, usdc, png, jpeg, m4a, mp3, and wav files.
Support
- 3ds Max version 2022, with latest update as Public Beta, plus version 2023. Includes Import from USD and Export to USD.
- Autodesk Fusion 360 supports USD export since April 2022.
- Blender includes support for USD export. Import support from version 3.0.
- Cinema 4D from Maxon includes support for USD for import and export.
- Houdini includes an implementation of USD, for purposes of format interchange and scene editing.
- Autodesk Inventor – supports USD as of version 2023.
- Maya – supports USD as of version 2022.
- mesh2mesh (macOS) converts USD to and from other formats.
- Nvidia has announced support for USD in Omniverse, a graphics collaboration platform.
- Davinci Resolve includes support for importing USD.
- Apple’s SceneKit supports .usdz files for 3D model interchange.
- Vectorworks 2023 Service Pack 3 onwards includes USD export and import
- Rhinoceros 3D through support is currently limited to v8 WIP (work in progress) and is not yet supported on the current stable release.
- Adobe Substance 3D Stager supports USD export.
[Content Source: https://en.wikipedia.org/wiki/Universal_Scene_Description]
USD ecosystem updates
What is USD
USD is a set of libraries and file formats that allows for assembly interchange and organization of any number of assets into scenes, shots or virtual objects, from object capture, asset creation layout and editing to rendering quick thumbnails for sharing or feedback to creating usdz assets for use in augmented reality or AR, the flexibility of USD can be a core component of an extensible collaborative 3d pipeline.
USD is a foundational technology that with the growing and deepening integration into digital content creation tools, is enabling more and more ways to creating assets and content, render, collaborate for games, AR, film, all with USD at the center.
Why USD
- Enables creative high-value content workflows
- Powerful 3D pipeline with collaboration support
- Rapidly emerging key workflow technology
- Built on 3D and production experience
USD was developed by Pixar to enable them to create the enormously complex movies we love. We at Apple worked with Pixar a few years ago to develop the usdz format to utilize usd in a single file to deliver rich AR content directly to your devices.
USD based workflows are highly collaborative enabling you to work independently on parts of an asset, easily incorporating your changes with others, it’s inherently extensible by design and is rapidly emerging as a key workflow technology. It’s built on decades and decades of production experience in the film industry and is increasingly being adopted for games, simulation, ar, manufacturing, and e-commerce.
Active USD community
- Open Source
- USD forum
- Academy Software Foundation
Pixar is using an open and collaborative model that the usd project has been following, the usd forum is active and full of contributors answering queries and the academy software foundation has a working group that meets regularly to gather input from interested groups.
3D file formats
The most basic format is obj which often essentially contains just a single 3d model, it has limited support for materials and no support for animations.
Then there’s a large group of more modern file formats, these usually support multiple models that can be laid out in a scene graph and have varying levels of support for material and definitions, may are tied to proprietary tools.
USD supports all of this and is additionally designed to be highly scalable, pixar developed usd for its use in films, so representing millions of objects is the typical case, and usd is built with collaboration as a core feature, allowing for many artists to work on a single scene without getting in each other’s way.
USDZ is the archive package for USD and inherits most of these features, is optimized for sharing and allows a single file to contain all the resources needed for rich AR interchange.
USD to USDZ
- Transition to USDZ for distribution and AR
- Tailored to target device
- Reality Converter and USDZ tools
USD naturally allows creation of USDZ files for AR content, but it’s also the front end for a full featured pipeline that delivers USDZ for AR and e-commerce, and assets for offline rendering. Last year, we released reality converter. A standalone utility to allow easy creation of a self-contained USDZ, from collections of USD files, obj, alembic, and a wide variety of other 3d formats as well. On the Apple developer site, there are command line tools to allow you to automate processing of batches of assets. As the delivery format for AR, usdz can be authored to target a specific apple device, letting you tailor the experience for your audience.
Growing USD ecosystem
The USD ecosystem is growing rapidly, it’s really exciting to see the things growing from the roots of what usd provides. Multiverse from j-cube brings the full power of non-destructive usd editing to the Mac, enabling insanely fast multi-dcc workflows as well as editing in the multiverse usd standalone editor, all powered by the metal hydra storm renderer.
Otoy’s octane renderer can plug into dccs like houdini and harnesses the blazing fast power of the gpu to render spectrally correct images directly from usd files.
zbrush works directly with usd mesh data to bring all the power of sculpting to a usd workflow.
And maya continues to integrate usd deeper and deeper, now using usd hydra rendering directly in the vp2 viewport.
Maya USD
- Seamless USD integration in Maya 2022
- Blendshape export
- USDZ texture import
Maya 2022 is the first version to have native seamless integration of USD. This means you can load gigabytes of usd data in seconds. And work directly with that data using maya’s native tools, usd data is integrated right into the attribute editor and the outliner, it brings all the usd benefits right inside the dcc, non-destructive editing variants and collaboration to name a few, apple worked with autodesk to improve the usd importing and exporting functionality as well, including blend shape support and using textures directly in usdz files.
USD schemas
- Collaboration with community
- Extend and improve schemas
- Schema proposals
Of course usd isn’t a static technology, apple is collaborating with pixar and the rest of the usd community to extend and improve usd support for the latest features in 3d content creation. Last year, apple, nvidia, pixar jointly developed a schema to represent properties needed for physics simulation, it can represent properties from real-time ar physics all the way to the offline simulations for robotics and film. It’s exciting to see such widespread support and so many great updates to the usd ecosystem. Another key feature of usd is hydra.
Hydra Storm in Preview
What’s Hydra
- USD’s modern rendering architexture
- Optimized for every large scenes
- Flexible delegates
Hydar is usd’s high performance modern rendering architecture designed specifically to get millions of objects to your screen as fast as possible. A core feature of its architecture allows it to flexibly handle different renderers.
Hydra Storm in Preview
- Render and feature consistency with other platforms
- Handle complex, production grade assets
- 3D scene toolset
- Quick Look
Hydra storm is one such renderer that is part of core usd open source distribution. We’re thrilled to share that preview and quick look in mac os will include a metal accelerated hydra storm renderer. This allows anyone to quickly view usd files in a way that’s consistent with hydra storm on other platforms all without having to open a dcc. It can handle complex production-grade assets and has some 3d scene exploration tools built in.
Here’s a layout asset from pixar’s toy story 4, the antiques mall, rendering in real time in preview, it has tens of thousands of individual objects, and about 34 million triangles. It’s a great example of production quality asset, and it’s a real challenge not only for the renderer but for also any scene navigation tools.
To help move around the scene, any cameras in the usd file are shown in the thumbnail views to the left letting me click through them, as well as the normal camera controls, I can click on objects in the view, and lock the camera to them.
To get a more detailed picture of the composition of the scene, we added an outliner that displays all of the object nodes, if I select an object in the view, then it’s highlighted in the outliner, but as you see from this huge list here we needed to provide better tools to assist with scenes of this complexity, so we’ve also added the ability to search object by name.
For example, if I search for the jukeboxes in the scene, I’ll get a list of them and can focus on each in turn, once I found the one I’m interested in, it can be saved as a separated usd file, and then I can view this again in preview and quick look, and use it in my ar scene in reality composer.
Features
- Camera thumbnails
- Scene graph view
- Search
- USD export
- Movie export
- Image export
On top of the features I’ve just shown you, preview also enables you to render animated usd scenes to a movie file, or render the current view to an image complete with alpha channel.
USDZ in Preview
- AR first format
- Rendered with RealityKit
- Consistent with AR Quick Look
For AR, preview uses the reality kit rendering engine to view usds files. It’s the same renderer as ar quick look on iOS, and is a great way to check what an on-device view will look like.
Metal support for USD
- Full Metal support for Hydra Storm
- Collaboration with Pixar Animation Studios
- Expected in Open Source by end of 2021
We’ve worked with pixar animation studios to add metal and m1 acceleration support to their powerful gpu renderer for producing 3d movies like soul and the upcoming movie luca. Pixar animation studios is making metal accelerated hydra storm available for integration in your apps through their usd open source project by the end of 2021.
Example USD workflow
Traditional workflow
In a traditional 3d workflow, sharing data between dccs is cumbersome and error prone, lengthy import and export steps are required to move data from one tool to another, this traditional workflow is also oneway, as only the last stages of the pipeline can see the overall scene, earlier stages lack context and how their contributions will be used.
USD workflow
In contrast, usd allows us to work in parallel using different tools and to work natively on the same data, crucially since all of our data is represented as usd, the import export step is eliminated and sharing is automatic.
A usd workflow enables faster iteration, better collaboration and opens up new and more creative possibilities.
Since it’s morning somewhere we can assume that it’s time for breakfast, so let’s make some pancakes. Here’s a sketch of how we’d like our overall scene to look.
We’ve storyboarded a few different concepts camera angles and layouts.
And decided this is our favorite.
Let’s see how we can use usd to turn this sketch into reality.
Structure
Since we’re using usd, we can arrange the scene into several separate files, each file will contribute to some aspect of the scene, be it the layout of the models, sculpting of the geometry or any animation or lighting we might want to add. These files are all referenced or linked into the main scene. This is the core feature of the usd which allows any number of people or tools to collaborate on the same scene at the same time.
So first, let’s set the table, we’re using autodesk maya to construct our overall scene, we need a usd file to store all of the layout data, this is where we assemble all of our assets and place them into the scene.
We also want a cool camera move, so we add a usd file to our scene for the camera animation
To create the 3d model of pancakes, we’ll use object capture on our real pancakes, after we’ve captured the model, we’ll use Pixologic’s zbrush to do some cleanup and make some enhancements. Here we’ve added a sculpting usd file to the scene where this data will live.
Can’t have pancakes without syrup, we make our virtual syrup in side effects houdini using its fluid simulation capabilities and native support for usd.
Finally, we’ll also use houdini to add some virtual lights to our scene.
Let’s see a little more detail how all of this comes together.
Object Capture
First we use object capture to create a 3d model from our real physical pancakes using a series of photographs taken from different angles, we can construct a 3d mesh and surface material of the pancakes.
Once we’ve acquired this model we can view it in preview. Here’s a breakdown of the captured model with and without textures. We’ll use this as a basis of our sculpting usd that we’ll edit later in zbrush.
While we’re capturing the pancakes, we can use maya to construct the initial set of usd files where all of our data will live.
Maya scene creation
As of maya 2022, usd is natively supported, and it’s easy to create a setup like this.
Once we set up our files and linked in our captured pancakes, we can do the layout, here I’ve assembled some of the props that make up the scene, I just need to make a few more adjustments.
This is the breakdown of our scene after we’ve done the layout while object capture produces excellent results we weren’t able to capture all of the angles of our pancakes before we ate them. So let’s take them into zbrush for some digital enhancement.
Using zbrush, we can paint in the missing geometry at the bottom of the stack of pancakes where they rested on the plate when we photograph them, in addition, we can add even physical details to the model as well as make some improvements to the surface material. These edits are being made in the sculpting usd file we created earlier, since the main scene is referencing this file, everyone on the team will get these updates as they are made.
Object capture looks great, but you can see the improvement that zbrush brings by adding detail and polish to the model while they already look scrumptious. They could use just a little bit more sweetening. For this task we’ll take our into houdini and add our virtual syrup and lighting.
In houdini, we can directly load all of our usd files as native data, houdini has powerful procedural tools to generate all sorts of effects and we can use it to pour virtual syrup all over our stack of pancakes.
Here is what our scene looks like with contributions from maya, zbrush, and houdini. Everything is brought together and now we just need to share it.
Sharing
Fortunately we’re using USD and we have a lot of options. We can create a usdz file and share our scene in augmented reality using ar quick look on iOS. On the mac, we can use the new built-in features of preview. Or we can use a studio level offline renderer to produce a photo reel video.
First we’ll bring our pancakes into ar. Reality composer lets use load our usd scene and make any final adjustments before we save it as an ar enabled usdz file.
Let’s load this up on the iPad pro and have a look.
Next let’s check out our scene in preview on the mac.
Finally we’ll create a video using otoy’s octane to create an offline path trace render of our scene. Whatever changes we make to adjust the lighting or materials can be shared back in ar or viewed in preview, usd makes this all possible.
From sketch to reality, thanks to usd and some great tools we’ve turned real pancakes into virtual breakfast.
[Content Source: (wwdc2021 – USD) https://www.youtube.com/watch?v=0Mb6_TwUkb0]