Future-proof & storage-efficient data-handling workflow

jbri

New time-lapser
#1
While I have read other posts about how to create a performant workflow (thanks @Aaron Priest) or another post where people tend to delete their RAW files after they rendered a video output, my question is:

What is a future-proof time lapse data-handling workflow?

At the moment I like to play around with quite intense color gradings. It might be that in the future I will prefer something a lot (!) less punchy/not crushing the blacks/virtual matte-look, for example, etc. Color depth? If the a7x produces 14Bit, the ideal codec should also have these 14Bit? I think: Timelapse takes a lot of time/effort. Displays will become better all the time. Maybe in 10 years I will have a head mounted display with “a few hundred k” (after 4k, 8k etc. become obsolete) and I don’t want to bake-in all my edits!

  • I don’t want to keep thousands of RAW files. Sony a7Riii = 43MB per File, 10s clip@25fps*43MP = 10.75GB.
  • Converting them to DNG gives me about 20% file size reduction, 8.6GB
  • Inspired by Aaron Priest’s workflow, I tried uncompressed 16Bit TIFF, after seeing 253MB per frame (!) I neglected that idea. In theory: 63GB
  • 8K CineForm uncropped encoding took about an hour. It seems to be wasted effort to me to let AE do the RAW processing AGAIN. Result: about 50% of the file size (14.1GB RAW photos vs. 7.27GB CineForm 8k uncropped).
  • The AE export to h265 took about 3h of rendering time (for an 11s sequence!) and resulted in about 200MB of uncropped 8K material.
    • Will I - in theory - be able to chop a Cineform 12bit 8K uncropped video into single dngs with 12bit and edit those in Lightroom, again?
  • Just as a comparison: LRTimelapse might give me a 600MB file (at HQ setting, 4k, h.265)
    • if I convert that using the Handbrake “Apple 2160p30fps” preset, that dwindles down to less than 20MB.
Alternatively, should I render one “flat profile” (so just lifting the shadows and bringing down the highlights) sequence as h.265 and that's future-proof enough to play with a lot of grading down the road?

From a theoretical viewpoint, we should be able to achieve a much higher compression because: Especially in a classic timelapse with clouds moving or even with subtle pan/tilt/slides the information should not change dramatically from picture to picture. If we save every single shot as a RAW/jpeg/... "all" the information has to be represented in EVERY picture. Whereas in an ideal video codec only the few percents of change from shot to shot have to be stored.

Editing/Intermediate vs Delivery codec?
  • ProRes/CineForm will require less computational effort while using it in a NLE (such as Premiere/AE/Final Cut) but this is traded off against file size.
  • If we can find a codec that fulfils all the requirements mentioned before but hasn't got such a high performance, I feel that this is OK. The initial idea was to reduce file size by replacing the storage of hundreds/thousands of RAWs by a video coded. In case heavy editing is required down the road (in a few years) a) performance of computers or the cloud will be higher b) proxy Workflow c) transcode intended codec to prores or whatever >>> so if there is something like a 12/14Bit h265 codec that generates small file sizes but causes long waiting times in a NLE, I don't mind, please let me know!

Thanks for any kind of input.
 

about us

Time Lapse Network is the Worldwide community dedicated to time-lapse photography.

Learn the technique, share your experiences and master it watching the best videos for free!

© 2013-18 Time Lapse Network - Created by Marco Famà

submit your video

Follow every single step in this tutorial and get your video straight to the home-page
Top