Intro -Guardian of the Image -- Bits and Bytes For the Cinematographer -Principals and underpinnings rather than

prescriptions. -Fundamental information that won't become obsolete as technologies change -- maybe you can even lead the technology change using some of these principals. -There are so many current and possible future permutations of not only cameras and formats but of ways to rate the cameras and subsequent pipelines. Amongst other things, this seminar should provide some tools that are not specic to today's technology only for evaluating which attributes of an image come from which step in the pipeline. Not lumping these things together and/or misattributing them. What is a digital image? -Just the images subdivided into pixels -- each pixel is three numbers from zero to one. That's all. -Effectively (though sometimes indirectly, since there is often an implied transform) a digital image is a numeric address system to look up all the possible states of an image sensor or a display device. -An image le is literally just a three-array that you can do any math on as such -- does not even have to be thought of as an image. An image could be a table and you could use a spreadsheet instead of an image editor to edit it. -RGB is the system we care most about -- we effectively use always use it, even when the underlying system is different. And all implemented systems are still merely this same kind of lookup. -There need to be enough elements in the image array not to see pixels and there needs to be enough steps from zero to 1 to not see increments. In other words: both must be subdivided nely enough to for the increments to be invisible and to appear smooth. -There is the conventional description in image science of "scene referenced" versus "display referenced," but this has a limited value (let's come back to it when we talk about density characteristics). The reason the concept is is tenuous to me is that, on the one hand, it is based entirely on the premise that a mathematical expression exists and is known that can convert between image le values and actual light values on set, but on the other hand, that the user cannot implement this expression. The distinction made more sense when video cameras had shading and caused clipping to the sensor signal and whatnot. -What attributes reside in the camera/sensor, which in the in-camera processing, and

which in the le? We'll have to examine this after we understand the attributes themselves, but it's important to understand the attributes of each individually and not mix up a part with the whole or a part with another part. Why We're Using Nuke For This Seminar -It is full quality and "agnostic," has direct pixel access. -It's mathematic transparency goes with the philosophy of the seminar: for image transforms not to obfuscated or made to sound magical. Bit Depth/Quantization -Breaking a smooth gradient into steps. Enough steps not to see increments and to appear perfectly smooth. -Bit depth is not latitude. -Bit depth is like resolution for color depth: not extensive, but intensive. You need to subdivide the image into enough pieces to not see the components and maintain the illusion of smoothness. -Conventional bit depth sizes and naming conventions (including per-channel versus per-pixel). Density characteristics: -the general types: logarithmic, rec709 (or gamma encoded), and linear light. Linear light and rec709 are NOT the same. -How general types differ from all the specic avors. -Implied/expected transforms: gamma, log/rec709, etc. -Quantization allocation. Linear is untenable as a storage format. Should only be used for calculations, not for storage. Show the allocation chart and give the measuring stick analogy. -A very concrete example of why linear is not a storage format: The way that ArriRaw is unpacked to linear light goes to around 36 on a scale of 0-1. -Float representation is inherently logarithmic. However, it puts much of its energy into fractions (smaller increments than the smallest integer quanta) and into negative numbers. So, the net after the gain and the loss is a wash and it's still inefcient as a camera le. -Densities characteristics (at least those that don't have horizontal lines in the graph)

can be inverted -- so the decision on density characteristics for storage should be made based on quantization allocation and coherence. -Using the descriptive term "tone map." -What is a LUT: fundamentally, conventionally, philosophically (including the philosophical and technical differences between a LUT and a grade). Compare ArriLUT to YedLUT for Alexa. -DP must understand relation of light in front of lens to sensor response, then to mapping of camera format, then to mapping for viewing. To optimize the camera and to accurately predict the resulting image after it goes through the pipeline, each step in the process must be known and understood discreetly, without collapsing several steps. We'll look at this again after each successive step. -Merely saying that it looked good on some set monitor is meaningless. -Any initial densities can be pushed to lmic densities, but NOT using lift/gamma/gain. (Including the math of lift/gamma/gain). -Let's get into some specic tests on cameras that can be done to understand this. -Now we can see why scene-referenced versus display referenced isn't really an either/ or dichotomy as much as an emphasis on one of two active functions. It may have been more meaningful with old style video cameras, which, unlike digital cinema cameras, used to have all kinds of image processing that included area shading and data clipping that could not be inverted. (Also, analog signal processing prevented inverting using math expressions rather than hardware.) Compression -Uncompressed has a certain xity: if you dene your specications, then the data size is xed (the size of a DPX or TIFF le is predictable and meaningful). Compression untethers this xity. -Throwing data away and hoping no one notices. By its very denition, compressed formats do not meet their own spec (otherwise they wouldn't be compressed.) So, compression is a method of throwing data away. -Compression formats are perceptual. So, you can't tell how good they are by the numbers. -There are two kinds of spatial compression (DCT and wavelet), and there's temporal compression. -The problem with them being perceptual is that they may work for a simple test but fall apart for color correction, for green screen keying, and/or for use as a master format for

subsequent compression layers. -Successive compression steps are cumulative -- it's the equivalent of duplication generations in analog. -Some are more open than others -- publishing the compression/decompression algorithms or keeping them only for business partners. Red does not let you transcode TO its codec, so you can't test the codec separate from the camera. -Coda: separating attributes of various steps in the pipeline. Resolution versus Apparent Sharpness -MTF versus resolution -Photosite count is only one factor in resolution. (Including compression falsifying resolution count -- let's look at down-rezing versus compressing.) -Green channel dominance and color subsampling. -The fact that sharpening is used is proof that subpixel resolution is quackery. -Real example where something called "5k" is only a tiny bit more resolute than "2k." -At high nominal resolutions, how things are scaled and/or sharpened has more to do with the look than true resolution does. -Noise and lm grain and perceptual sharpness. -Coda: separating attributes of various steps in the pipeline. Bayer vs RGB and "Raw" nonsense -What the two systems are. -As we saw, green is most important for resolution, so using the real and limited surface area and circuitry with green weight makes sense in a camera. And RGB makes sense for le formats to manipulate: it resolves the spacial issues that will eventually need to be addressed with Bayer. So, it's not an issue of one being better -- it's just complex. -Confusion of naming conventions. Bayer and RGB are both good systems but vendors use the confusion about the difference to cloud the meaning of specs. Notice that when we say "2k" or "4k" we almost never say the units -- "k" is not a unit, it means "thousand." But Bayer and RGB are different units. -"Raw" is a red herring. "Not-yet-debayered" is not necessarily a benet. "Raw" is meant to evoke mystical adulation, but it's still a computer le with numbers in it. It

doesn't mean you magically have access to more data than the le can store. "Raw" is not a magical incantation that makes a computer le no longer subject to realities of how computer les work. -Some "raw" formats are also uncompressed. Uncompressed is indeed an advantage over compressed, but the fact that some les are both raw and uncompressed is not necessarily an advantage. Video Range / Full Range -It's too bad that this convention exists and we have to cover it but it is one of the most pervasive pitfalls for monitoring and workow. It's so pervasive that this almost accidental thing is actually one of the most practically important things in the seminar. -Coda: separating attributes of various steps in the pipeline. Spectral Response -Let's not go too deep into this -- it's a little off topic, but we'll explain the basics. Analogy to sound waves summing up the multitudinous components into a single amplitude. -This is the complexity that cannot be inverted. Once many wavelengths have been summed together and coded as single (or three) amplitudes, inverting is not possible. 3-channel data can be inverted to other 3 channel data (as long as there is no clipping/ clamping), but it cannot be inverted to original multi-wavelength components. -Coda: separating attributes of various steps in the pipeline. Testing Camera And Formats -Separating noise, latitude, rating, of a camera. -Formats: pushing and prodding on full quality images with full quality software. Including re-compressing. -Let's also think of some tricks to try to isolate variables when a device doesn't want you to (like if there's a camera that has only compressed output with no option for uncompressed). Thinking About Real Workows and Not Falling For Big Numbers -If you don't think about every step, it doesn't work. Both for practicality and for quality. (A higher quality format can yield a worse end-result if you're not properly prepared to handle the format downstream.) Review Some Specic Devices: File Formats, Software, Cameras

-image sequences versus clips -wrappers versus media streams -DPX, EXR, TIFF, ProRes4444, ProResHQ, R3D, XAVC, H.264 -Nuke, Shake, Adobe, Color Correctors -Alexa, Film, Red Epic. -Also color interchange: ICC, LUT, CDL. Some Fallacies About Real Workows: -"colorbaked" nonsense -compression and VFX: both le handling and keying issues -we're still not there for "ofine is online" to really work -- for multiple reasons. Some lmic attributes for color correction: -Toe and Shoulder -Crossover -Red goes towards yellow -- red is the most idiosyncratic primary. (Maybe partially because it's a wider swath of the visible wavelengths and because it's the color that wraps around.) -Max saturation is achieved earlier but has lower overall density. (Low density high saturation is what gives the impression of low saturation.) Politics: Communicating With Technicians And Creative People -Speaking to emphasize a common goal, not departmental or individual needs.