Page 2 of 3
Re: 4K HDR framerate lag on latest official build
Posted: 05 Sep 2019, 23:07
by davilla
Everbrave wrote: 05 Sep 2019, 08:59
Is the next build eminent, I.e. within few days, say next week?
BTW: selecting Video Toolbox delivers washed-out colors for HDR contents; this is not the case for AVFoundation, however, so what is the reason for this behavior?
The reason is videotoolbox uses GLES for rendering and there is no tone mapping of HDR -> SDR.
Since AVFoundation codec can handle HDR -> SDR, adding tone mapping to videotoolbox is low priority.
Re: 4K HDR framerate lag on latest official build
Posted: 06 Sep 2019, 09:42
by Everbrave
davilla wrote: 05 Sep 2019, 23:07
Everbrave wrote: 05 Sep 2019, 08:59
Is the next build eminent, I.e. within few days, say next week?
BTW: selecting Video Toolbox delivers washed-out colors for HDR contents; this is not the case for AVFoundation, however, so what is the reason for this behavior?
The reason is videotoolbox uses GLES for rendering and there is no tone mapping of HDR -> SDR.
Since AVFoundation codec can handle HDR -> SDR, adding tone mapping to videotoolbox is low priority.
So, what is Video Toolbox doing that AVFoundation doesn’t do? Low-level HW Access? Isn’t AVFoundation using Video Toolbox as well?
Re: 4K HDR framerate lag on latest official build
Posted: 07 Sep 2019, 16:48
by davilla
VideoToolBox is a video codec. It takes video demux packets and decodes them into video frames of yuv (8 or 10 bit, depending on source content). It's the caller's job to render those video frames. So MrMC takes those video frames and renders them into onto a NSView with GLES, which is then displayed on iOS/tvOS.
When VideoToolBox is selected as the codec, MrMC uses GLES to do the rendering. GLES can only do 8 bit, so HDR -> SDR but since tone mapping is not done, you get washed out colors. MrMC would have to render in Metal to get 10 bit HDR rendered with correct colors.
AVFoundation codec actually uses an API called AVSampleBufferDisplayLayer. It's the same thing that AVPlayer (apple's player) uses. It uses VideoToolBox for decode and Metal internally to decode AND render to a "layer" which we associate with a NSView. This NSView is under the GLES layer (which is transparent). So what you see is video with GLES over top.
Calling AVFoundation codec CDVDVideoCodecAVFoundation is actually over general. But calling it CDVDVideoCodecAVSampleBufferDisplayLayer looked funny so we named it CDVDVideoCodecAVFoundation
Re: 4K HDR framerate lag on latest official build
Posted: 07 Sep 2019, 22:02
by Everbrave
davilla wrote: 07 Sep 2019, 16:48
VideoToolBox is a video codec. It takes video demux packets and decodes them into video frames of yuv (8 or 10 bit, depending on source content). It's the caller's job to render those video frames. So MrMC takes those video frames and renders them into onto a NSView with GLES, which is then displayed on iOS/tvOS.
When VideoToolBox is selected as the codec, MrMC uses GLES to do the rendering. GLES can only do 8 bit, so HDR -> SDR but since tone mapping is not done, you get washed out colors. MrMC would have to render in Metal to get 10 bit HDR rendered with correct colors.
AVFoundation codec actually uses an API called AVSampleBufferDisplayLayer. It's the same thing that AVPlayer (apple's player) uses. It uses VideoToolBox for decode and Metal internally to decode AND render to a "layer" which we associate with a NSView. This NSView is under the GLES layer (which is transparent). So what you see is video with GLES over top.
Calling AVFoundation codec CDVDVideoCodecAVFoundation is actually over general. But calling it CDVDVideoCodecAVSampleBufferDisplayLayer looked funny so we named it CDVDVideoCodecAVFoundation
Now I understand, thanks a lot
Re: 4K HDR framerate lag on latest official build
Posted: 08 Sep 2019, 19:31
by davilla
@LeoNatan, you are clueless about how video is decoded and rendered on iOS/tvOS
The issue is with AVSampleBufferDisplayLayer, Apple changed its default behavior and you have to add magic sauce to restore the original behavior. AVPlayer works because they (dh'u) have added the magic sauce to AVPlayer. But they neglected to tell 3rd party developers that use AVSampleBufferDisplayLayer. Typical
Re: 4K HDR framerate lag on latest official build
Posted: 16 Sep 2019, 12:30
by Samekas
Same problem here... Any way to downgrade version?
Re: 4K HDR framerate lag on latest official build
Posted: 16 Sep 2019, 12:42
by amet
Samekas wrote: 16 Sep 2019, 12:30
Same problem here... Any way to downgrade version?
MrMC ? No
but...
debuglog might have answers why you have issues
Re: 4K HDR framerate lag on latest official build
Posted: 16 Sep 2019, 16:57
by davilla
Samekas wrote: 16 Sep 2019, 12:30
Same problem here... Any way to downgrade version?
Fixed in MrMC 3.9.2 testflight. Will be released sometime this week.
Re: 4K HDR framerate lag on latest official build
Posted: 23 Sep 2019, 07:30
by Samekas
davilla wrote: 16 Sep 2019, 16:57
Samekas wrote: 16 Sep 2019, 12:30
Same problem here... Any way to downgrade version?
Fixed in MrMC 3.9.2 testflight. Will be released sometime this week.
Awesome, thank you!
Re: 4K HDR framerate lag on latest official build
Posted: 23 Sep 2019, 07:36
by MrSpidey
davilla wrote: 16 Sep 2019, 16:57
Samekas wrote: 16 Sep 2019, 12:30
Same problem here... Any way to downgrade version?
Fixed in MrMC 3.9.2 testflight. Will be released sometime this week.
Thank you very much.
As I am not on testflight (and assuming this works correctly), what do you think would be a reasonable time we would see it as an update to everyone else?