How To produce max quality / res & smaller files in post production

chasg

New time-lapser
#1
Hi All,

I need a bit of guidance, I hope someone can help.

I'm no expert when it comes to AfterEffects and PremierePro, and I need to use both to do what I want to do, which is to create 3 files from my RAWs:
1) a lossless full resolution (5K to 8K) high quality video file to use as a base for further processing (e.g. stabilisation or deflickering). This because I understand that using it for a base will be faster than rendering the RAW sequence every time
2) a 4K file for stock sites (with effects applied)
3) a watermarked HD for uploading to Instagram and Vimeo

I'm an experienced TL photographer (https://vimeo.com/bothhemispheres), and entirely comfortable with Lightroom and LRTimelapse, it's just the steps afterwards that I need help with.

Here's what I'm attempting to do now, and the problems I'm having (I'd be very grateful for any insights you more experienced post-production people might be able to offer):
1) edit in LR and LRT, save XMP with raw files (my cameras produce anywhere from 5K to 8K files)
open sequence in AfterEffects, create comp from sequence

2) render sequence as quicktime 16-bit 100% quality photo-jpeg (I'm on a PC, so no ProRes for me).
-a recent 2000 shot 8K TL took just over 3 hours for this

3) re-import that .mov file into AE, create 4K comp and apply effects (usually Flicker Free, and sometimes Warp Stabilise), render again with 16-bit 100% quality photo-jpeg (this one for stock sites).
-this can take a _really_ long time, depending, I'm guessing, on the effects. But...last night I set a render going and I came back this morning to find that it'd been rendering for 10 hours, with 54 more hours estimated to go (umm... I must've made some horrendous mistake somewhere).

4) create a watermarked HD file in Premiere (I have a good workflow set up for watermarks there). This is where things fall down, and I don't know how to solve them. What I thought was the right thing to do was:
a) create HD comp in AE using 4K .mov file (resize .mov file to comp size)
b) Dynamic Link in PP to that HD AE comp
c) edit that HD comp in PP

Here's the problem: using that linked AE HD comp in PP, it takes minutes to render any frame (no quick scrubbing possible), so I can't practically make any edits (Mostly all I want to do is simple fade in and outs with watermark text, plus fade from and to black, all of my Vimeo clips do this).

I've tried creating really small proxies in AE, but that doesn't really speed things up in PP at all, much to my surprise.

So my problems are twofold:
1) That's a lot of rendering, any tips on streamlining the processes? It'd be nice to set something up so that it could run overnight.
2) I can't seem to work in PP with the AE comps as dynamic links (I have a very good PC: fast processor, plenty of RAM, m.2 scratch disk). I must be doing something wrong, but I can't figure out what.

Many thanks for any insights (which I hope are useful for any future readers of this thread).

Chas
 

Aaron Priest

Active time-lapser
#2
Here is where my workflow differs a little, after a lot of testing to see what is faster on my computers (your results might vary depending on hardware):

I don't bring RAW files into After Effects. It just drags horribly on performance. I convert to 8-bit JPEG or 16-bit TIFF at full resolution first (depending if I need 10-bit or 12-bit intermediates, or if 8-bit will be fine with no color grading down the line). Out of all the methods of converting RAW files to either JPEG or TIFF, I've found Lightroom's export to be the most multithreaded and fastest. So I edit as you do with LRTimelapse and Lightroom, but export via Lightroom to 16-bit TIFF (I don't use LRTimelapse's export module, although you could--it uses the same Lightroom export and then renames the files and opens LRTimelapse for you).

If you want ProRes on a PC, you could render these 8-bit JPEGs or 16-bit TIFFs to 10-bit ProRes with LRTimelapse. It's very quick, much faster than After Effects. I like the control I have in After Effects though, and I render to 12-bit Cineform as my intermediate or "master" file. I usually render this at 8K for more flexibility down the line. I delete the temporary TIFF or JPEGs when I'm done with my project and happy with my final render.

This 8K 12-bit Cineform master file will play very nicely in Premiere Pro for any further editing, such as scaling down to 4K or 1080p, adding a watermark or music, or using the clip in a much longer video of multiple clips. I don't use Dynamic Link back to After Effects, that is also horribly slow in performance.

The initial export from Lightroom to 16-bit TIFFs or 8-bit JPEGs for temporary files is the step that takes the longest for me, particularly if you have several thousand frames. Rendering with After Effects isn't as bad. Everything else down the chain from that flies.

Hope that helps!
 

chasg

New time-lapser
#4
Hey Aaron, we know each other from the RamperPro facebook page (I'm "Jete" there), thanks very much for the considered reply! As @marcofama says, your feedback is incredibly valuable (heck, I even built my PC based on original guidance from you :)

Great info, and it all makes sense, though it does seem to me that rendering a set of jpegs or tiffs is an extra step (although I'm very happy to be proven wrong). As I see it, there are two paths in AE to a master file:

1) render intermediates in LR, then assemble in AE and render a master movie
2) render a master movie directly from RAWs in AE​

Does the RAW processing in AE add that much overhead that number 2 is faster than 1?​

LOL, I had thought I'd left behind rendering an intermediate JPEG or TIFF long ago, once raw + xmp rendering became possible (this was before I was shooting timelapse, but rather when I was a stills event photog). Fair enough, I can work that way if it's lots faster (I like lots faster!). I've rendered over a hundred timelapses via LRT before I started looking at AE and PP, so that long step will be familiar (phew!).

Your recommendation for Cineform is just what I need: I was pretty desperate to determine the most suitable codec for my full res master file (I had thought it should be photo-jpeg).

Re: Prores on PC. I had heard that the PC implementation of Prores, via ffmpeg, wasn't "real" enough for stock sites like Getty, which is why I was avoiding it. Getty accepts photo-jpeg movies from PCs (and Prores from Macs), so that'll be my 4K codec. It'll be h264 for Vimeo and Instagram (same HD file).

I'm sorry the AE to PP Dynamic Link is so slow, I really do like working with them in the stills world (to and from Photoshop, InDesign, Illustrator, LR).

Lastly: you say you like the control you get in AE for rendering vs LRT, could you elaborate a bit (if you've got time) on what that control would be?

Thanks again!

Chas
 

chasg

New time-lapser
#5
Aaron, one more question: I use two effects in AE, Warp Stabilise and Flicker Free (deflicker plugin, as the name implies:) At what step do you recommend I apply those? On the jpeg/tiff photo series before rendering the cineform master movie, or on that master movie itself, before rendering to 4K and HD?
 
#7
Thank you for your answer Aaron! ;)

BTW can you give me the link to buy the exact Cineform codec version that you are using for your master files please? I am looking for it since a long time and can't find it since it has been bought by GoPro...

Thank you!
 

Aaron Priest

Active time-lapser
#8
Hey guys! Just catching up here after a workshop. Let me work down the list, hehe...

There are pretty much four paths to consider here, try each one and see which yields you the fastest results, if you time each one with a stop watch and then add up all the accumulated time:
  1. Export RAWs from Lightroom to 16-bit TIFF (after editing via LRTimelapse, etc.), use the 16-bit TIFFs in AE to render to 4K or 8K 12-bit CineForm as in intermediate. Use the 12-bit intermediate in Premiere Pro for rendering an actual video with titles, music, etc. to either another 12-bit CineForm or H.264/H.265 for web use. Or just send the AE 12-bit intermediate to a client if all they require is a single clip. In the past, this has been my fastest workflow. I need to test all this again with the latest 2017 CC builds on my new desktop to verify that. Let me know how you make out.
  2. Use the edited RAW files and their .xmps in AE directly and let AE do the RAW conversion when creating a 12-bit CineForm. In my previous tests with anything newer than AE 2014 this was no longer multithreaded and slower at converting than LR’s export. I need to test again with the latest version of AE and see if it has improved.
  3. Use the edited RAW files and their .xmps in AE, but don’t render. Include the AE project in a Premiere Pro project via dynamic link on the timeline. This is the slowest performance of all from my previous testing, even with AE 2014, as it dramatically slows down your timeline scrubbing and editing in PP while making a finished movie. I’d rather work with far more efficient 12-bit CineForm (or 10-bit ProRes) intermediates in Premiere Pro than wait on dynamic link to re-render previews on every edit!
  4. The last option is to export from LR to 8 or 16-bit TIFFs and render to 10-bit ProRes with LRTimelapse, bypassing AE entirely. It’s one of the easiest methods of getting ProRes on a PC. It uses ffmpeg, which I’ve also heard is not a pure ProRes implementation like QuickTime generates on a Mac. I’m not certain how different or how close it is, or what issues there might be as a result. ProRes in general has both gamma and color shift issues to begin with and so I avoid the format when possible: http://www.drastic.tv/support-59/supporttipstechnical/202-prores-colour-shifts-in-post-production
The last option does not give you the control you have with After Effects, regarding your crop to 16:9 or 16:10, or using any form of pan/zoom (unless you do that within your edits, which is often not as a smooth as doing it in AE). You also have more options for frame blending, motion blur, and time remapping, although LRT’s more basic implementation of them does an excellent job. You could export out of AE to a full resolution 24MP, 36MP, 42MP, etc. CineForm intermediate (width needs to be a multiple of 16) and have more flexibility in Premiere Pro for pan/zoom, but I find performance is much faster downscaling to 4K or 8K within AE before exporting. You could even downscale your 16-bit TIFF exports out of Lightroom to 2160px wide before starting your AE project if you knew you weren’t doing any pans/zooms and simply crop to your desired 4K or UHD ratio in AE. Downsized Lightroom exports will go a little faster with a dense sensor like a D810 or a7R II. Also, skip zip compression on the TIFFs and let them be uncompressed for faster exporting. Since you are deleting the temp files anyway after rendering your intermediate, there is no long term cost in disk space. I also copy my RAW files to my 960 Pro PCIe NVMe drive as a new catalog (export as a new catalog and open) before exporting to TIFF so all of that happens on my fastest drive. I’ll delete the temp files when I’m done and move the finished AE project and 12-bit CineForm to my RAID array, which is fast enough for video rendering. I just want to get that painful RAW conversion done as fast as possible.

If you use warp stabilize, extra deflickering, etc. that is better done in AE before exporting as a 12-bit CineForm, vs. doing it on Premiere Pro down the chain. I find that LRTimelapse’s deflickering on the RAW files is so good that I rarely need to do anything later. I do go through LRT’s visual deflicker process 3 or 4 times to smooth it out though, which takes a while to generate. It will go faster if you export your RAW files to a new catalog on a PCIe NVMe SSD drive first as I mentioned above. Make sure you import your edits back from the temporary catalog to your main catalog when you are done, to preserve your edits. I don’t bother with that extra catalog step for small projects, only big ones where I know I’ll be using visual deflickering quite a lot (long holy grails, etc.). I’ll just work off my RAID array of hard drives for smaller projects of 1000 frames or less (like star trails), it’s fast enough.

You don’t have to buy anything to use CineForm in Creative Cloud, it’s native! However, finding it in After Effects is confusing; the dialog boxes don’t match up at all with Premiere Pro or Media Encoder. Here is how you get YUV 4:2:2 at 10 bits per channel or RGBA 4:4:4:4 at 12 bits per channel: https://helpx.adobe.com/after-effects/using/gopro-cineform-codec-settings.html Usually 10-bit 422 is good enough if you aren’t doing any color grading down the line. Since we do our main editing on RAW files in LR/LRT, we usually don’t need 12-bit 4444 for timelapse clips, unless you want to deliver a client the absolute highest quality for an intermediate. If you have a slower computer, try 10-bit 422 for snappier Premiere Pro performance. If you are going to render to H.264/H.265 for web use anyway without any color grading, you won’t notice a difference. If you are using 8-bit JPEGs instead of 16-bit TIFFs then there is really no gain exporting to 12-bit CineForm.

Lastly, regarding color profiles, make sure you set your AE project to 16-bit per channel and HDTV (Rec. 709) 16-235 to get the color mapping correct. If you are using JPEGs, you can leave it at 8-bit. If your final destination is 10-bit H.265 you could try Rec. 2020 or Rec. 2100 for a wider color gamut, but not many players/TVs support either of these profiles yet. Best to play with on your own setup with Plex media server, etc. for your specific TV model.
 

about us

Time Lapse Network is the Worldwide community dedicated to time-lapse photography.

Learn the technique, share your experiences and master it watching the best videos for free!

© 2013-18 Time Lapse Network - Created by Marco Famà

submit your video

Follow every single step in this tutorial and get your video straight to the home-page
Top