Welcome, Guest. Please login or register.

Author Topic: Video overlay - essential for fast video playback  (Read 10521 times)

Description:

0 Members and 1 Guest are viewing this topic.

Offline PiruTopic starter

  • \' union select name,pwd--
  • Hero Member
  • *****
  • Join Date: Aug 2002
  • Posts: 6946
    • Show only replies by Piru
    • http://www.iki.fi/sintonen/
Video overlay - essential for fast video playback
« on: June 27, 2012, 08:23:40 PM »
Recently there has been a lot of discussion about overlay, and how it supposedly is obsolete and/or of small priority.

It is neither. Overlay is absolutely critical for low-end systems displaying video. The benefits of overlay are two-fold:

  • Offloading the video scaling to hardware.
  • Offloading the YCbCr -> RGB conversion to hardware.
The first point is the most critical. It allows the video to be scaled to whatever size requested without requiring the application to perform the frame scaling with the CPU. Even if you choose not to resize the video output at all, much of the video content (such as DVDs) require scaling out of the box to display the video with the correct aspect ratio.

The second point removes the need for the application to convert the YCbCr encoded image information to RGB understood by the graphics system. This leaves more time for actual video and audio decoding.

The practical implications are that if the OS or graphics driver doesn't support overlay, video playback will suffer. This is especially critical with the low-end systems that already spend considerable resources decoding the video and audio in the first place.

So neither is overlay low priority nor obsolete.

PS. It does not matter how the overlay is implemented. It could be dedicated overlay mode supported by the graphics hardware, or it could be implemented via the 3D hardware. The important point is that both OS and the device driver implement it and the APIs providing the overlay to the applications.

PPS. Overlay has other uses as well, such as in emulators where the graphics can be stretched to whatever size needed. Without it you're stuck with the postage stamp on a 1920x1080 display.
« Last Edit: June 29, 2012, 09:12:18 PM by Karlos »
 

Offline PiruTopic starter

  • \' union select name,pwd--
  • Hero Member
  • *****
  • Join Date: Aug 2002
  • Posts: 6946
    • Show only replies by Piru
    • http://www.iki.fi/sintonen/
Video overlay - essential for fast video playback
« Reply #1 on: June 28, 2012, 10:29:25 AM »
Recently there has been a lot of discussion about overlay, and how it supposedly is obsolete and/or of small priority.

It is neither. Overlay is absolutely critical for low-end systems displaying video. The benefits of overlay are two-fold:

1. Offloading the video scaling to hardware.

2. Offloading the YCbCr -> RGB conversion to hardware.

The first step is the most critical. It allows the video to be scaled to whatever size requested without requiring the application to perform the frame scaling with the CPU. Even if you choose not to resize the video output at all, much of the video content (such as DVDs) require scaling out of the box to display the video with the correct aspect ratio.

The second point removes the need for the application to convert the YCbCr encoded image information to RGB format understood by the graphics system.

The practical implications are that if the OS or graphics driver doesn't support overlay, video playback will suffer. This is especially critical with the low-end systems that already spend considerable amount of resources decoding the video and audio in the first place.

So neither is overlay low priority nor obsolete.

PS. It does not matter how the overlay is implemented. It could be dedicated overlay mode supported by the graphics hardware, or it could be implemented via the 3D hardware, as long as it is.

PPS. Overlay has other uses as well, such as in emulators where the graphics can be stretched to whatever size needed. Without it you're stuck with the postage stamp on a 1920x1080 display.


MODS: I think I posted this yesterday as an misc announcement by accident. Please nuke that post.
 

Offline woof

  • Jr. Member
  • **
  • Join Date: Feb 2003
  • Posts: 94
    • Show only replies by woof
    • http://uae.is.free.fr
Re: Video overlay - essential for fast video playback
« Reply #2 on: June 28, 2012, 01:39:21 PM »
Hello

>1. Offloading the video scaling to hardware.
But the existing CompositeTags() can do hardware resizing on x1000...

>2. Offloading the YCbCr -> RGB conversion to hardware.
But Altivec can (may) do that...

So when when you say "we need overlay" what exactly is needed ? a full gxvideo.library implementation ? or just a function to convert/display an YCbCr buffer ?

Alain Thellier
 

Offline LaserBack

  • Sr. Member
  • ****
  • Join Date: Mar 2007
  • Posts: 345
    • Show only replies by LaserBack
Re: Video overlay - essential for fast video playback
« Reply #3 on: June 28, 2012, 02:04:09 PM »
yes overlay is essential
on the Amiga only the Picasso 4 and the cybervision 64 supports overlay
on winuae have not been implemented yet...Tony was asked to implement it on  winuae for fast and better video play back but he said he is not interested in add that function to winuae...he said that everyone must use windows to watch videos
 

Offline Crumb

  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 1786
  • Country: 00
    • Show only replies by Crumb
    • http://cuaz.sourceforge.net
Re: Video overlay - essential for fast video playback
« Reply #4 on: June 28, 2012, 03:11:14 PM »
Quote from: woof;698233
Hello

>1. Offloading the video scaling to hardware.
But the existing CompositeTags() can do hardware resizing on x1000...

>2. Offloading the YCbCr -> RGB conversion to hardware.
But Altivec can (may) do that...

So when when you say "we need overlay" what exactly is needed ? a full gxvideo.library implementation ? or just a function to convert/display an YCbCr buffer ?

Alain Thellier


These *PIP* functions expect a driver with working overlay:
http://amiga.sourceforge.net/amigadevhelp/phpwebdev.php?action=List&funcgroup=Picasso96 otherwise current videoplayers won't be able to open a PIP/overlay windows and will have to switch to other rendering methods.
The only spanish amiga news web page/club: Club de Usuarios de Amiga de Zaragoza (CUAZ)
 

Offline spirantho

Re: Video overlay - essential for fast video playback
« Reply #5 on: June 28, 2012, 04:51:02 PM »
My understanding is that Overlay is a hardware device - it's where the graphics card renders its frame buffer and then renders a seperate area (the overlay).

Using 3D to emulate overlay is a perfectly valid way of doing it, but it's not actually overlay - which is Hans' point.

Overlay is obsolete.
Using 3D to gain the same benefits as overlay is not.

Hans isn't saying that the drivers don't need video support, he's saying it should be done via 3D, not overlay. Which is correct, as overlay is indeed pretty much obsolete these days of 3D graphics cards.
--
Ian Gledhill
ian.gledhill@btinternit.com (except it should be internEt of course...!)
Check out my shop! http://www.mutant-caterpillar.co.uk/shop/ - for 8-bit (and soon 16-bit) goodness!
 

Offline Crumb

  • Hero Member
  • *****
  • Join Date: Mar 2002
  • Posts: 1786
  • Country: 00
    • Show only replies by Crumb
    • http://cuaz.sourceforge.net
Re: Video overlay - essential for fast video playback
« Reply #6 on: June 28, 2012, 04:58:31 PM »
Quote from: spirantho;698251
Hans isn't saying that the drivers don't need video support, he's saying it should be done via 3D, not overlay.

And OS4 should support various cpu cores and include memory protection but it doesn't and most probably won't.

Quote
My understanding is that Overlay is a hardware device - it's where the graphics card renders its frame buffer and then renders a seperate area (the overlay).

There's an API to handle overlay/PIP or whatever name you want to use. The graphic card drivers should take care of providing an implementation for that interface. It doesn't matter how you implement overlay, what matters is if it's supported or not because all amiga video players from the nineties to 2012 use PIP/overlay API.

Hans has said gpu is very complex and you can expect creating working 3d drivers will take much more time. If you plan to implement overlay compatibility after 3d drivers are done it will take much more time. Permedia2 lacked true overlay and CGX coders implemented Overlay support using a pair of textures. It didn't require full OpenGL support.
« Last Edit: June 28, 2012, 05:01:38 PM by Crumb »
The only spanish amiga news web page/club: Club de Usuarios de Amiga de Zaragoza (CUAZ)
 

Offline itix

  • Hero Member
  • *****
  • Join Date: Oct 2002
  • Posts: 2380
    • Show only replies by itix
Re: Video overlay - essential for fast video playback
« Reply #7 on: June 28, 2012, 05:05:18 PM »
Quote
But the existing CompositeTags() can do hardware resizing on x1000...

It is faster use an overlay. Besides, when you are setting up a call to CompositeTags() or BltBitMapScaleArgs() overlay has done it already.

Quote
But Altivec can (may) do that...

It is faster use an overlay. Besides, low end machines dont have AltiVec.

Overlay is not obsolete.
My Amigas: A500, Mac Mini and PowerBook
 

Offline Heiroglyph

  • Hero Member
  • *****
  • Join Date: Jun 2010
  • Posts: 1100
    • Show only replies by Heiroglyph
Re: Video overlay - essential for fast video playback
« Reply #8 on: June 28, 2012, 07:32:26 PM »
Once you've converted to RGB, the video takes much more memory and bandwidth.

Yuv (or ycbcr) is skipping many color components.

RGB is 4:4:4 (for every 4 pixels, there are 4 sames of each component)
Yuv can be 4:2:2, 4:1:1, 4:2:0 (really a misnomer), etc. so it's much smaller in memory.

DVD, BluRay and all the common mp4/h264 variants are 4:2:0 for example.

Not having to convert to a larger more cumbersome RGB format is one of the biggest benefits of overlay, 3D "faked" or not.
 

Offline Karlos

  • Sockologist
  • Global Moderator
  • Hero Member
  • *****
  • Join Date: Nov 2002
  • Posts: 16879
  • Country: gb
  • Thanked: 5 times
    • Show only replies by Karlos
Re: Video overlay - essential for fast video playback
« Reply #9 on: June 28, 2012, 09:07:28 PM »
I think part of the problem is that many people are confusing the concept with with a particular implementation of the concept. To me, and probably to Hans, a video overlay is a very specific hardware implementation that is distinct from chromakeying and video texturing. To the next guy, maybe not. So, for the sake of clarity, I'll use the term "hardware video surface" to describe the general concept of a hardware assisted mechanism for rendering YCbCr formatted frames without the need for software colour conversion or scaling.

Nobody is arguing against the need for "hardware video surface" support in display drivers. Even when you don't need to scale the output, not having to do the colour conversion can make a difference in the amount of data that has to be transferred over the bus as the YCbCr data is typically more compact than RGB. In a perfect world, various parts of the hardware that are involved in video stream decoding would be openly documented and we could use those too.

The argument has been over the merits of particular ways of implementing support for "hardware video surfaces". The original hardware mechanism that we are used to from the days of Cirrus Logic and S3 chipsets of yesteryear (and variations thereof) are what are being described as obsolete here. I haven't looked too closely, but from what I gather, this particular method doesn't even exist in a lot of current generation cards. Instead, video colour conversion and scaling is handled by parts of the hardware 3D pipeline through the use of texture mapping, in which each YCbCr frame is used as a texture and rendered onto a quad (a rectangle primitive, optionally realized as a triangle fan or strip on hardware without direct support for quads).

Right now, as far as I know no specific implementation exists for the RadeonHD drivers. With adequate 3D support, however, implementing a hardware video surface via video texturing becomes feasible. Since both 3D and video playback are desirable features, it makes sense to focus on the overlap.
int p; // A
 

Offline Akiko

  • Hero Member
  • *****
  • Join Date: Jun 2002
  • Posts: 1026
    • Show only replies by Akiko
Re: Video overlay - essential for fast video playback
« Reply #10 on: June 28, 2012, 09:26:23 PM »
The good news is thanks to funding this guy appears to be working on these drivers full time. I'm sure many of the features including those right up to what the latest new Radeon graphics cards have to offer with video playback enhancements will come. :)
 

Offline magnetic

  • Hero Member
  • *****
  • Join Date: Feb 2002
  • Posts: 2531
    • Show only replies by magnetic
Re: Video overlay - essential for fast video playback
« Reply #11 on: June 29, 2012, 02:34:53 AM »
Piru

Thanks for the technical explanation. I guess the fanboys that DONT have overlay needed this. I found some of the comments about not having it being ok to be funny. Overlay makes a big difference in many practical ways that affect all users.

Dont know if this is really a "news" item but glad for the clarification to stop silly additional  posts
bPlan Pegasos2 G4@1ghz
Quad Boot:Reg. MorphOS | OS4.1 U4 |Ubuntu GNU-Linux | MacOS X

Amiga 2000 Rom Switcher w/ 3.1 + 1.3 | HardFrame SCSI | CBM Ram board| A Squared LIVE! 2000 | Vlab Motion | Firecracker 24 gfx

Commodore CDTV: 68010 | ECS | 9mb Ram | SCSI -TV | 3.9 Rom | Developer EPROMs
 

Offline Akiko

  • Hero Member
  • *****
  • Join Date: Jun 2002
  • Posts: 1026
    • Show only replies by Akiko
Re: Video overlay - essential for fast video playback
« Reply #12 on: June 29, 2012, 02:48:42 AM »
A discussion about overlay is classed as a news item?
 

Offline PiruTopic starter

  • \' union select name,pwd--
  • Hero Member
  • *****
  • Join Date: Aug 2002
  • Posts: 6946
    • Show only replies by Piru
    • http://www.iki.fi/sintonen/
Re: Video overlay - essential for fast video playback
« Reply #13 on: June 29, 2012, 03:19:25 AM »
Quote from: Akiko;698300
A discussion about overlay is classed as a news item?
The mistake was mine, I originally posted to a wrong category by accident. This "news item" should be deleted. To quote the actual discussion thread post:
Quote
MODS: I think I posted this yesterday as an misc announcement by accident. Please nuke that post.
« Last Edit: June 29, 2012, 03:48:16 AM by Piru »
 

Offline Akiko

  • Hero Member
  • *****
  • Join Date: Jun 2002
  • Posts: 1026
    • Show only replies by Akiko
Re: Video overlay - essential for fast video playback
« Reply #14 on: June 29, 2012, 08:44:14 PM »
It goes to show how active the mods are here now days, talk about a slippery slope.