←back to thread

Pixar's Render Farm

(twitter.com)
382 points brundolf | 2 comments | 02 Jan 21 19:56 UTC | HN request time: 0.588s | source
Show context
thomashabets2 ◴[02 Jan 21 20:38 UTC] No.25616274[source]▶
I'm surprised they hit only 80-90% CPU utilization. Sure, I don't know their bottlenecks, but I understood this to be way more parallelizable than that.

I ray trace quake demos for fun at a much much lower scale[0], and have professionally organized much bigger installs (I feel confident in saying even though I don't know Pixar's exact scale).

But I don't know state of the art rendering. I'm sure Pixar knows their workload much better than I do. I would be interested in hearing why, though.

[0] Youtube butchers the quality in compression, but https://youtu.be/0xR1ZoGhfhc . Live system at https://qpov.retrofitta.se/, code at https://github.com/ThomasHabets/qpov.

Edit: I see people are following the links. What a day to overflow Go's 64bit counter for time durations on the stats page. https://qpov.retrofitta.se/stats

I'll fix it later.

replies(5): >>25616362 #>>25616369 #>>25616380 #>>25616401 #>>25617648 #
1. brundolf ◴[02 Jan 21 20:47 UTC] No.25616369[source]▶
My guess would be that the core-redistribution described in the OP only really works for cores on the same machine. If there's a spare core being used by none of the processes on that machine, a process on another machine might have trouble utilizing it because memory isn't shared. The cost of loading (and maybe also pre-processing) all of the required assets may outweigh the brief window of compute availability you're trying to utilize.
replies(1): >>25617718 #
2. thomashabets2 ◴[02 Jan 21 23:32 UTC] No.25617718[source]▶
Yeah. With my pov-ray workload the least efficient part is povray loading the complex scene. And that's not multithreaded. The solution that works for me is that I can just start ncore frames concurrently, or just two, but stagger them a bit (thus there's always one frame doing parallelizable work and can use all cores, even if the other is not).

But at that point it may get into RAM constraint, or some as yet unmentioned inter-frame dependency/caching.