Basically though, Pixar doesn't have the scale to make custom chips (the entire Pixar and even "Disney all up" scale is pretty small compared to say a single Google or Amazon cluster).
Until recently GPUs also didn't have enough memory to handle production film rendering, particularly the amount of textures used per frame (which even on CPUs are handled out-of-core with a texture cache, rather than "read it all in up front somehow"). I think the recent HBM-based GPUs will make this a more likely scenario, especially when/if OptiX/RTX gains a serious texture cache for this kind of usage. Even still, however, those GPUs are extremely expensive. For folks that can squeeze into the 16 GiB per card of the NVIDIA T4, it's just about right.
tl;dr: The economics don't work out. You'll probably start seeing more and more studios using GPUs (particularly with RTX) for shot work, especially in VFX or shorts or simpler films, but until the memory per card (here now!) and $/GPU (nope) is competitive it'll be a tradeoff.
High end VFX/CG usually tessellates geometry down to micropolygon, so you roughly have 1 quad (or two triangles) per pixel in terms of geometry density, so you can often have > 150,000,000 polys in a scene, along with per vertex primvars to control shading, and many textures (which can be paged fairly well with shade on hit).
Using ray tracing pretty much means having all that in memory at once (paging sucks in general of geo and accel structures, it's been tried in the past) so that intersection / traversal is fast.
Doing lookdev on individual assets (i.e. turntables) is one place where GPU rendering can be used as the memory requirements are much smaller, but only if the look you get is identical to the one you get using CPU rendering, which isn't always the case (some of the algorithms are hard to get working correctly on GPUs, i.e. volumetrics).
Renderman (the renderer Pixar use, and create in Seattle) isn't really GPU ready yet (they're attempting to release XPU this year I think).
There's a huge I/O bottleneck as well as you're reading huge textures (I've seen textures as big as 1 TB) and writing constantly to disk the result of the renderer.
Other than that, most of the tooling that modern studios use is off the shelf, for example, Autodesk Maya for Modelling or Sidefx Houdini for Simulations. If you had a custom architecture then you would have to ensure that every piece of software you use is optimized / works with that.
There are studios using GPUs for some workflows but most of it is CPUs.
I'd imagine another reason is that Pixar uses off-the-shelf Digital Content Creation apps (DCCs) like Houdini and Maya in addition to their proprietary software, so while they could optimize some portions of their pipeline, it's probably better to develop for more general computing tasks. They also mention the ability to "ramp up" and "ramp down" as compute use changes over the course of a show
[1] https://devmesh.intel.com/projects/supercharging-pixar-s-ren...
[2] https://nvidianews.nvidia.com/news/pixar-animation-studios-l...
https://graphics.pixar.com/library/SoulRasterizingVolumes/pa...
On a personal note, I had a pretty visceral "anti-" reaction to the movie Soul. I just felt it too trite in its handling of themes that humankind has wrestled with since the dawn of time. And jazz is probably the most cinematic of musical tastes. Think of the intros to Woody Allen's Manhattan or Midnight in Paris. But it felt generic here.
That said the physically based rendering is state of the art! If you've ever taken the LIE toward the Queensborough Bridge as the sun sets across the skyscraper canyons of the city you know it is one of the most surreal tableaus in modern life. It's just incredible to see a pixel perfect globally illuminated rendering of it in an animated film, if only for the briefest of seconds ;)
Can you speak to any competitive advantages a vfx-centric gpu cloud provider may have over commodity AWS? Even the RenderMan XPU looks to be OSL / Intel AVX-512 SIMD based. Thanks!
Supercharging Pixar's RenderMan XPU™ with Intel® AVX-512
So that can easily end up being several hundred gigabytes of source image data. At rendertime, only the textures that are needed to render what's visible in the camera are loaded into memory and utilized, which typically ends up being a fraction of the source data.
Large scale terrains and environments typically make more use of procedural textures, and they may be cached temporarily in memory while the rendering process happens to speed up calculations
A GPU enabled version of RenderMan is just coming out now. I imagine their farm usage after this could change.
https://gfxspeak.com/2020/09/11/animation-studios-renderman/
I’m purely speculating, but I think the main reason they haven’t been using GPUs until now is that RenderMan is very full featured, extremely scalable on CPUs, has a lot of legacy features, and it takes a metric ton of engineering to port and re-architect well established CPU based software over to the GPU.
This is probably true today, but leaves the wrong impression IMHO. The clear trend is moving toward GPUs, and surprisingly quickly. Maya & Houdini have release GPU simulators and renderers. RenderMan is releasing a GPU renderer this year. Most other third party renderers have already gone or are moving to the GPU for path tracing - Arnold, Vray, Redshift, Clarisse, etc., etc.
And GPU rendering has been gaining momentum over the past few years, but the biggest bottleneck until recently was availabe VRAM. Big budget VFX scenes can often take 40-120 GB of memory to keep everything accessible during the raytrace process, and unless a renderer supports out-of-core memory access, then the speed up you may have gained from the GPU gets thrown out the window from swapping data
This is no longer true, and hasn’t been for around a decade. This is a left-over memory of when GPUs weren’t using IEEE 754 compatible floating point. That changed a long time ago, and today all GPUs are absolutely up to par with the IEEE standards. GPUs even took the lead for a while with the FMA instruction that was more accurate than what CPUs had, and Intel and other have since added FMA instructions to their CPUs.
From a different perspective, think about supercomputers - many supercomputers do indeed do relatively specific things (and I would assume some do run custom hardware), but the magic is in the interconnects - getting the data around effectively is where the black magic is.
Also, if you aren't particularly time bound, why bother? FPGAs require completely different types of engineers, and are generally a bit of pain to program for even ignoring how horrific some vendor tools are - your GPU code won't fail timing for example.
I disagree with this takeaway. But full disclosure I’m biased: I work on OptiX. There is a reason Pixar and Arnold and Vray and most other major industry renderers are moving to the GPU, because the trends are clear and because it has recently become ‘worth it’. Many renderers are reporting factors of 2-10 for production scale scene rendering. (Here’s a good example: https://www.youtube.com/watch?v=ZlmRuR5MKmU) There definitely are tradeoffs, and you’ve accurately pointed out several of them - memory constraints, paging, micropolygons, etc. Yes, it does take a lot of engineering to make the best use of the GPU, but the scale of scenes in production with GPUs today is already firmly well past being limited to turntables, and the writing is on the wall - the trend is clearly moving toward GPU farms.
But, very soon after I left Lucas, ILM started pushing ray tracing a lot harder. Getting good quality results per ray is very difficult. Much easier to throw hardware at the problem and just cast a whole lot more rays. So, they moved over to being heavily GPU-based around that time. I do not know the specifics.
An entire movie at 2k, uncompressed floating point rgb would be about 4 terabytes.
So I'm well aware of the trade offs. As I mentioned, for lookdev and small scenes, GPUs do make sense currently (if you're willing the pay the penalty of getting code to work on both CPU and GPU, and GPU dev is not exactly trivial in terms of debugging / building compared to CPU dev).
But until GPUs exist with > 64 GB RAM, for rendering large scale scenes, it's just not worth it given the extra burdens (increased development costs, heterogeneous sets of machines in the farm, extra debugging, support), so for high-end scale, we're likely 3/4 years away yet.
Uncompressed, it’s 93Gb of render data, plus 130Gb of animation data if you want to render the entire shot instead of a single frame.
From what I’ve seen elsewhere, that’s not unusual at all for a modern high end animated scene.
(I'm aware of the OSL batching/GPU work that's taking place, but it remains to be seen how well that's going to work).
From what I've heard from friends in the industry (at other companies) who are using GPU versions of Arnold, the numbers are no-where near as good as the upper numbers you're claiming when rendering at final fidelity (i.e. with AOVs and Deep output), so again, the use-cases - at least for high-end VFX with GPU - are still mostly for lookdev and lighting blocking iterative workflow from what I understand. Which is still an advantage and provides clear benefits in terms of iteration time over CPU renderers, but it's not a complete win, and so far, only the smaller studios have started dipping their toes in the water.
Also, the advent of AMD Epyc has finally thrown some competitiveness back to CPU rendering, so it's now possible to get a machine with x2 as many cores for close to half the price, which has given CPU rendering a further shot in the arm.
The trend is pretty clear, though. The size of scenes than can be done on the GPU today is large and growing fast, both because of improving engineering and because of increasing GPU memory speed & size. It’s just a fact that a lot of commercial work is already done on the GPU, and that most serious commercial renderers already support GPU rendering.
It’s fair to point out that the largest production scenes are still difficult and will remain so for a while. There are decent examples out there of what’s being done in production with GPUs already:
https://www.chaosgroup.com/vray-gpu#showcase
They are really expensive though. But chassis and rackspace also isn't free. If one beefy node with a couple GPUs can replace have a rack of CPU only Nodes the GPUs are totally worth it.
I'm not too familiar with 3D rendering but in other workloads the GPU speedup is so huge that if its possible to offload to the GPU it made sense to do it from a economical perspective.
Not only that, they are massive, kickout a whole bunch of heat in new and interesting ways. worse still they depreciate like a mofo.
the tip top renderbox of today is next years comp box. a two generation old GPU is a pointless toaster.
Are GPUs starting to be used at earlier points in the pipeline? Yes, absolutely, but they always were to a degree in previs and modelling (via rasterisation). They are gradually becoming more useable at more steps in pipelines, but they're not there yet for high-end studios.
In some cases, if a studio's happy using an off-the-shelf renderer with the stock shaders (so no custom shaders at all - at least until OSL is doing batching and GPU stuff, or until MDL actually supports production renderer stuff) studios can use GPUs further down the pipeline, and currently that's smaller scale stuff from what I gather talking to friends who are using Arnold GPU. Certainly the hero-level stuff at Weta / ILM / Framestore isn't being done with with GPUs, as they require custom shaders, and they aren't going to be happy with just using the stock shaders (which are much better than stock shaders from 6/7 years ago, but still far from bleeding edge in terms of BSDFs and patterns).
Even from what I hear at Pixar with their lookdev Flow renderer things aren't completely rosy on the GPU front, although it is at least getting some use, and the expectation is XPU will take over there, but I don't think it's quite ready yet.
Until a studio feels GPU rendering can be used for a significant amount of the renders (that they do, for smaller studios, the fidelity will be less, so the threshold will be lower for them), I think it's going to be a chicken-and-egg problem of not wanting to invest in GPUs on the farms (or even local workstations).
Isn’t there a better Vray or Arnold comparison somewhere?
As in my summary comment, an A100 can now run real scenes, but will cost you ~$10k per card. For $10k, you get a lot more threads from AMD.
The folks at Framestore and many other shops already don’t do more than XX GiB per frame for their rendering. So for me, this comes down to “can we finally implement a good enough texture cache in optix/the community” which I understand Mark Leone is working on :).
The shader thing seems easy enough. I’m not worried about an OSL compiled output running worse than the C-side. Divergence is a real issue, but so many studios are now using just a handful of BSDFs with lots of textures to drive, that as long as you don’t force the shading to be “per object group” but instead “per shader, varying inputs is fine”, you’ll still get high utilization.
The 80 GiB parts will make it so that some shops could go fully in-core. I expect we’ll see that sooner than you’d think, just because people will start doing interactive work, never want to give it up, and then say “make that but better” for the finals.
An AMD Rome/Milan part will give you 256 decent threads on a 2S box with a ton of RAM for say $20-25k at list price (e.g., a Dell power edge without any of their premium support or lots of flash). By comparison, the list price of just an A100 is $15k (and you still need a server to drive the thing).
So for shops shoving these into a data center they still need to do a cost/benefit tradeoff of “how much faster is this for our shows, can anyone else make use of it, how much power do these draw...”. If anything, the note about more and more software using CUDA is probably as important as “ray tracing is now sufficiently faster” since the lack of reuse has held them back (similar things for video encoding historically: if you’ve got a lot of cpus around, it was historically hard to beat for $/transcode).
Is any of that true?
Note though these are not single 1TB textures, they’re multiple sets of textures, plus all of the layers that constitute them. Some large robots In particular had 65k 4K textures if you count the layers.
If 8k gaming becomes a real thing you can expect work to be done towards a solution, but until then not so much.
Edit: added [desktop] preceding PCIE
OTOH, Vfx rendering involves a varying scene with moving light sources, cameras, objects, textures, and physics. Much more dynamic interactions. This is a gross simplification but I hope it helps.
The reason is that those softwares need to be sold to many, and a big part of studios are doing advertise and series. GPU rendering is perfect for them as they don't need/can't afford large scale render farms.
About your example, that not honest. It's full of instances and perfect use case for a "Wow" effect but it's not a production shot. Doing a production shot required complexity management on the long run, even for CPU rendering. On this side, GPU is more "constrained" than CPU, management is even more complex.
I know it doesn't makes sense to tell your clients that what they are doing is nonsense, but if I saw something like that going on, the first thing I would do is chase down why it happened. Massive waste like that is extremely problematic while needing to make a sharper texture for some tiny piece that gets close to the camera is not a big deal.
Over texturing like this can be a good decision depending on the production. Asset creation often starts a long time before shots or cameras are locked down.
If you don’t know how an asset is to be used it makes sense to texture all of it upfront as if it will be full screen, 4K.
Taking an asset off final to ‘Upres’ it for a can be a pain in the ass and more costly than just detailing it up in the first place.
In isolation it’s a insane amount of detail and given perfect production planning it is normally not needed, but until directors lock down the scripts and shots it can be the simplest option.
This was easier to rely on in the days before ray tracing, when texture filtering was consistent because everything was from the camera. Ray differentials from incoherent rays aren't quite as forgiving.
> If you don’t know how an asset is to be used it makes sense to texture all of it upfront as if it will be full screen, 4K.
4k textures for large parts of the asset in the UV layout can be an acceptable amount of overkill. That's not the same as putting 65,000 4k textures on something because each little part is given its own 4k texture. I know that you know this, but I'm not sure why you would conflate those two things.
> Taking an asset off final to ‘Upres’ it for a can be a pain in the ass and more costly than just detailing it up in the first place
It is very rare that specific textures need to be redone like that and it is not a big deal.
650GB of textures for one asset drags everything from iterations to final renders to disk usage to disk activity to network usage down for every shot in a completely unnecessary way. There isn't a fine line between these things, there is a giant gap between that much excessive texture resolution and needing to upres some piece because it gets close to the camera.
> Asset creation often starts a long time before shots or cameras are locked down.
This is actually fairly rare.
> In isolation it’s a insane amount of detail and given perfect production planning it is normally not needed, but until directors lock down the scripts and shots it can be the simplest option.
That's rarely how the time line fits together. It's irrelevant though, because there is no world where 65,000 4k textures on a single asset makes sense. It's multiple orders of magnitude out of bounds of reality.
I am glad that you have that insane amount of scalability as a focus since you are making tools that people rely on heavily, and I wish way more people on the tools end thought like this. Still, it is about 1000x what would set off red flags in my mind.
I apologize on behalf of whoever told you that was necessary, because they need to learn how to work within reasonable resources (which is not difficult given modern computers), no matter what project or organization they are attached to.
Take for example, a large hero asset like King Kong.
Kong look development started many months before a script was locked down. Kong is 60ft tall, our leading lady is 5’2”.
We think we need shots where she’ll be standing in Kong’s hands, feet, be lifted up to his face, nose etc.
So we need fingers prints that will stand up at 4K renders, tear ducts, pores on the inside on the nose, etc etc but we don’t know. All of which will have to match shot plates in detail.
We could address each of these as the shots turn up and tell the director (who owns the company) he needs to wait a few days for his new shot, or you can break Kong into 500 patches and create a texture for each of the diffuse, 3 spec, 3 subsurface, 4 bump, dirt, blood, dust, scratch, fur, flow etc etc inputs to our shaders.
Let’s says we have 500 UDIM patches for Kong so we can sit our leading lady on the finger tips, and 20 channels to drive our shaders and effects systems.
When working the artist uses 6 paint layers for each channel ( 6 is a massive underestimate for most interesting texture work).
So we have 500 patches * 20 channels * 6 layers which gives us 60k images. Not all of these will need be at 4K however.
For Kong replace any hero asset where shots will be more placed “in and on” the asset rather than “at”. Heli carriers, oil rigs, elven great halls, space ships, giant robots.... The line between asset and environment is blurred at that point and maybe think “set” rather than “asset”
We both know that stuff isn't showing up on film and that the excess becomes a CYA case of the emperor's new clothes where no one wants to be the one to say it's ridiculous.
> When working the artist uses 6 paint layers for each channel ( 6 is a massive underestimate for most interesting texture work).
This is intermediary and not what is being talked about.
I would say it's the opposite. There is nothing necessary about 10,000 4k maps and definitely nothing typical. Workflows trade a certain amount of optimization for consistency, but not like this.
> patronizing who actually works in the industry.
I don't think I was patronizing. This person is valuable in that they are trying to make completely excessive situations work. Telling people (or demonstrating to them) they are being ridiculous is not his responsibility and is a tight rope to walk in his position.
> Also speaking as an authority that knows best
If I said that 2 + 2 = 4 would you ask about a math degree? This is an exercise in appeal to authority. This person and myself aren't even contradicting each other very much.
He is saying the extremes that he has seen, I'm saying that 10,000 pixels of texture data for each pixel in a frame is enormous excess.
The only contradiction is that he seems to think that because someone did it, it must be a neccesity.
Instead of confronting what I'm actually saying, you are trying to rationalize why you don't need to.
Usually the way VFX works is that technology (R&D) is very moved away from production. The artist job is getting the shot done regardless of technology and they have very short deadlines. They usually push the limits.
Digital artists are not very tech savvy in a lot of disciplines, it is not feasible to have a TD in the delivery deadlines of the shots for a show.
The person at Weta also told you how Weta actually worked in Kong which is very typical. You don't know upfront what you need. And you dismissed it as something unnecessary, still, is how every big VFX studio works. Do you feel that you know better and/or everyone is doing something wrong and hasn't really thought about it? If that is the case you might have a business opportunity for a more efficient VFX studio!
> how Weta actually worked in Kong which is very typical
It is not typical to have 10,000 4k maps on a creature. What has been typical when rendering at 2k is a set of 2k maps for face, torso, arms and legs. Maybe a single arm and leg can be painted and the UVs can be mirrored, though mostly texture painters will layout the UVs separately and duplicate the texture themselves to leave room for variations.
> it is not feasible to have a TD in the delivery deadlines of the shots for a show.
Actually most of the people working on shots are considered TDs. Specific asset work for some sequence with a hero asset is actually very common, which makes sense if you think about it from a story point of view of needing a visual change to communicate a change of circumstances.
4k rendering (was the 2017 king kong rendered in 4k?) and all the closeups of king kong mean that higher resolution maps and more granular sections are understandable, but it doesn't add up to going from 16 2k maps to 10,000 4k maps. Maps like diffuse, specular and subsurface albedo are also just multiplicative, so there is no reason to have multiple maps unless they need to be rebalanced against each other per shot (such as for variations).
You still never actually explained a problem or inconsistency with anything I've said.
> Actually most of the people working on shots are considered TDs.
That's not true in the studios I've been. TDs is usually reserved to more close to pipeline folks that aren't doing shot work (as in, delivering shots). They're supporting folks doing so.
For the record, I haven't downvoted you at all.
This was created with the requirement that the director be able to use it at will. Closeups. Set replacements, destruction, the works.
You don’t have a shot breakdown or camera list.
You’ve got 6 months of pre production to support 1000 shots. Once in production you will be the only texture artist supporting 30 TDS.
How do you spend your 6 months to make sure production runs smoothly?
I’m kinda interested in your experience of this stuff as the numbers you’re quoting for 2k work are, in my experience, waaaaaay off and are closer to how a high end games asset would currently be textured.
I don’t disagree with you that the numbers involved are crazy when taken in isolation but it is (or at least was 5 years ago) a very common workflow at ILM, Weta, Dneg, DD, R&H Framestore etc etc. The quoted high numbers are the very upper end but many thousands of assets on hundreds of productions have been textured at what I believe you would consider “insane” detail levels.
You clearly understand many of the issues involved but downplay the complexity in running high end assets in less than perfect production.
Unless the industry has changed dramatically in 5 years shot changes, per shot fixes, variants (clean, dirty, destroyed), shader tweaks, happen on every single show I’ve ever been part of.
Render time and storage is one factor as is individual artist iteration, but the real productivity killer is an inter discipline iteration.
Going from a “blurry texture” note in comp to a TD fix to a texture “upres” is potentially a 5 person, 4 day turn around. I would trade a whole bunch of cpu and storage to avoid that.
Computers are cheap, people are expensive, people coordinating even more so.