.... ain't gonna work with the current code-base of either xulrunner (firefox and ice* types) or webkit (chrome, midori, qupzilla). darn.
the reason for webkit is: although gstreamer supports vdpau and vaapi, and even can be activated to do hardware-accelerated decode, webkit (at least as best i can find) the gtk+ variant then targets *opengl* as the output rendering engine, rather than asks VAAPI / VDPAU to handle it directly on the webkit engine's behalf.
what that means is: with software-rendering OpenGL of the decoded pixels it's simply too much for the A20 to handle.
amazingly i have actually seen icecat do about 2 or 3 fps video decoding in software, i really don't know how it managed, and i haven't really been able to replicate it.
anyway if people are expecting videos to be played in web browsers, it's gonna take some work. for now it's possible to download them (youtube-dl or other methods) and then use a video player.
l.
--- crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
Hi Luke,
So, if I've understood this post correctly, video playback on vlc would perhaps be within the bounds of what the A20 hardware / software could do?
Russell Just a Fourier transform away from creating a computer game...
On 09/08/2016, Luke Kenneth Casson Leighton lkcl@lkcl.net wrote:
.... ain't gonna work with the current code-base of either xulrunner (firefox and ice* types) or webkit (chrome, midori, qupzilla). darn.
the reason for webkit is: although gstreamer supports vdpau and vaapi, and even can be activated to do hardware-accelerated decode, webkit (at least as best i can find) the gtk+ variant then targets *opengl* as the output rendering engine, rather than asks VAAPI / VDPAU to handle it directly on the webkit engine's behalf.
what that means is: with software-rendering OpenGL of the decoded pixels it's simply too much for the A20 to handle.
amazingly i have actually seen icecat do about 2 or 3 fps video decoding in software, i really don't know how it managed, and i haven't really been able to replicate it.
anyway if people are expecting videos to be played in web browsers, it's gonna take some work. for now it's possible to download them (youtube-dl or other methods) and then use a video player.
l.
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
arm-netbook mailing list arm-netbook@lists.phcomp.co.uk http://lists.phcomp.co.uk/mailman/listinfo/arm-netbook Send large attachments to arm-netbook@files.phcomp.co.uk
--- crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
On Tue, Aug 9, 2016 at 10:40 PM, Russell Hyer russell.hyer@gmail.com wrote:
Hi Luke,
So, if I've understood this post correctly, video playback on vlc would perhaps be within the bounds of what the A20 hardware / software could do?
yes. ffplay, vlc, mpv, mplayer - all work fine, that's what this update was about: https://www.crowdsupply.com/eoma68/micro-desktop/updates/cedrus
basically by using the full capabilities of the A20's accelerated hardware (both CEDAR and G2D), the CPU usage is somewhere around the 10% mark. peanuts in other words.
the reason why those work is, through vdpau plugins there's two parts: "decode" part and "actual on-screen display" part. all those video players have an option to do *both* things, and the vdpau-sunxi plugin that the Cedrus team wrote goes "okay, we're configured to use G2D, let's just go for it".
that's why you end up with the overlay issues, btw, because the G2D hardware is told to write direct to an area on-screen, bypassing the X11 protocol.
anyway, it's *ASSUMED* in the webkit and xulrunner engines that the "on-screen display" bit can be handled by OpenGL or... something-else-don't-know-don't-care-but-basically-it's-not.
and that means a bit of rewriting of both webkit and xulrunner so that they hand over the display bit to G2D. that's going to require some specialist work.
l.
thanks for the detailed answer.
russ sent from a non libre device
On Tuesday, 9 August 2016, Luke Kenneth Casson Leighton lkcl@lkcl.net wrote:
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
On Tue, Aug 9, 2016 at 10:40 PM, Russell Hyer <russell.hyer@gmail.com javascript:;> wrote:
Hi Luke,
So, if I've understood this post correctly, video playback on vlc would perhaps be within the bounds of what the A20 hardware / software could do?
yes. ffplay, vlc, mpv, mplayer - all work fine, that's what this update was about: https://www.crowdsupply.com/eoma68/micro-desktop/updates/cedrus
basically by using the full capabilities of the A20's accelerated hardware (both CEDAR and G2D), the CPU usage is somewhere around the 10% mark. peanuts in other words.
the reason why those work is, through vdpau plugins there's two parts: "decode" part and "actual on-screen display" part. all those video players have an option to do *both* things, and the vdpau-sunxi plugin that the Cedrus team wrote goes "okay, we're configured to use G2D, let's just go for it".
that's why you end up with the overlay issues, btw, because the G2D hardware is told to write direct to an area on-screen, bypassing the X11 protocol.
anyway, it's *ASSUMED* in the webkit and xulrunner engines that the "on-screen display" bit can be handled by OpenGL or... something-else-don't-know-don't-care-but-basically-it's-not.
and that means a bit of rewriting of both webkit and xulrunner so that they hand over the display bit to G2D. that's going to require some specialist work.
l.
arm-netbook mailing list arm-netbook@lists.phcomp.co.uk javascript:; http://lists.phcomp.co.uk/mailman/listinfo/arm-netbook Send large attachments to arm-netbook@files.phcomp.co.uk javascript:;
Hi Luke,
Am 10.08.2016 um 00:16 schrieb Luke Kenneth Casson Leighton:
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
On Tue, Aug 9, 2016 at 10:40 PM, Russell Hyer russell.hyer@gmail.com wrote:
Hi Luke,
So, if I've understood this post correctly, video playback on vlc would perhaps be within the bounds of what the A20 hardware / software could do?
yes. ffplay, vlc, mpv, mplayer - all work fine, that's what this update was about: https://www.crowdsupply.com/eoma68/micro-desktop/updates/cedrus
basically by using the full capabilities of the A20's accelerated hardware (both CEDAR and G2D), the CPU usage is somewhere around the 10% mark. peanuts in other words.
the reason why those work is, through vdpau plugins there's two parts: "decode" part and "actual on-screen display" part. all those video players have an option to do *both* things, and the vdpau-sunxi plugin that the Cedrus team wrote goes "okay, we're configured to use G2D, let's just go for it".
that's why you end up with the overlay issues, btw, because the G2D hardware is told to write direct to an area on-screen, bypassing the X11 protocol.
Just a minor correction. The G2D (mixer processor) hardware (only on A10/A20) is only used within libvdpau-sunxi to blit one rgba surface into another one. It has nothing to do with bringing something *really* to screen, because it only handles the blit or fill actions on rgba surfaces. The only use of rgba surfaces that i know of, is to realize an OSD, subtitles etc... as an overlay over the video.
NVidia and the others most likely do *real* blits of that rgba into the the yuv video. Maybe with the use of OpenGL. Our libvdpau-sunxi does separate these surfaces and hold them both on separater layers. The display enginge takes care of these two layers and pushes the video into layer 1 and then the OSD (if existent) into another layer 2 which lays over the video layer. So data keeps separated. This is the problem why some things with libvdpau-sunxi do not work exactly like someone expects. Especially in windowed mode. We use a colorkey to mask the video area and don't render with X11's mechanism, but bypass the whole X11 system and write directly to framebuffer as you said. So to sum up, overlay issues are caused by the display engine, or the way libvdpau-sunxi build his surfaces, but not by G2D. As we don't have G2D available on future Allwinner SoCs, the blits there are done in software using pixman.
anyway, it's *ASSUMED* in the webkit and xulrunner engines that the "on-screen display" bit can be handled by OpenGL or... something-else-don't-know-don't-care-but-basically-it's-not.
Oh. Maybe i completely misunderstood and my previous comment was b*****it, but is there really an o-s-d over the video within the browser window? If so, they most likely use something with OpenGL, yes.
and that means a bit of rewriting of both webkit and xulrunner so that they hand over the display bit to G2D. that's going to require some specialist work.
It would theoretically be possible to let libvdpau-sunxi use OpenGL/ES under the hood and blit everything into one surface/texture. But this maybe would drop the performance advantages compared to how we do know. And you depend on Mali.... There are some efforts to integrate/simulate the nv_gl_interop extension with libvdpau-sunxi and OpenGL/ES, which lets OpenGL use vdpau surface and vice versa. So it would be possible to let vdpau do the rendering and OpenGL/ES do the presentation. Problem is, that it's not finished and it's not intended by the spec anyway to either use VDPAU on ARM nor combine it with OpenGL/ES, as the nv_gl_interop only talks about i386 and OpenGL. <- This is the way kodi works with vdpau for example.
And the whole problem of this all is, that i currently cannot used with mainline kernel :) So i'd look further in having an open source video decoding driver and the needed display driver bits getting mainline ;)
Andreas
l.
arm-netbook mailing list arm-netbook@lists.phcomp.co.uk http://lists.phcomp.co.uk/mailman/listinfo/arm-netbook Send large attachments to arm-netbook@files.phcomp.co.uk
--- crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
On Wed, Aug 10, 2016 at 8:23 AM, Andreas Baierl list@imkreisrum.de wrote:
Hi Luke,
Am 10.08.2016 um 00:16 schrieb Luke Kenneth Casson Leighton:
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
On Tue, Aug 9, 2016 at 10:40 PM, Russell Hyer russell.hyer@gmail.com wrote:
Hi Luke,
So, if I've understood this post correctly, video playback on vlc would perhaps be within the bounds of what the A20 hardware / software could do?
yes. ffplay, vlc, mpv, mplayer - all work fine, that's what this update was about: https://www.crowdsupply.com/eoma68/micro-desktop/updates/cedrus
basically by using the full capabilities of the A20's accelerated hardware (both CEDAR and G2D), the CPU usage is somewhere around the 10% mark. peanuts in other words.
the reason why those work is, through vdpau plugins there's two parts: "decode" part and "actual on-screen display" part. all those video players have an option to do *both* things, and the vdpau-sunxi plugin that the Cedrus team wrote goes "okay, we're configured to use G2D, let's just go for it".
that's why you end up with the overlay issues, btw, because the G2D hardware is told to write direct to an area on-screen, bypassing the X11 protocol.
Just a minor correction. The G2D (mixer processor) hardware (only on A10/A20) is only used within libvdpau-sunxi to blit one rgba surface into another one. It has nothing to do with bringing something *really* to screen, because it only handles the blit or fill actions on rgba surfaces. The only use of rgba surfaces that i know of, is to realize an OSD, subtitles etc... as an overlay over the video.
interesting.... because if i don't enable OSD i get absolutely nothing (black screen) on mpv and other players.
NVidia and the others most likely do *real* blits of that rgba into the the yuv video. Maybe with the use of OpenGL. Our libvdpau-sunxi does separate these surfaces and hold them both on separater layers. The display enginge takes care of these two layers and pushes the video into layer 1 and then the OSD (if existent) into another layer 2 which lays over the video layer. So data keeps separated. This is the problem why some things with libvdpau-sunxi do not work exactly like someone expects. Especially in windowed mode. We use a colorkey to mask the video area and don't render with X11's mechanism, but bypass the whole X11 system and write directly to framebuffer as you said.
.... i knew it was _something_ like that... :)
So to sum up, overlay issues are caused by the display engine, or the way libvdpau-sunxi build his surfaces, but not by G2D. As we don't have G2D available on future Allwinner SoCs, the blits there are done in software using pixman.
let's hope there's enough horsepower in those!
anyway, it's *ASSUMED* in the webkit and xulrunner engines that the "on-screen display" bit can be handled by OpenGL or... something-else-don't-know-don't-care-but-basically-it's-not.
Oh. Maybe i completely misunderstood and my previous comment was b*****it, but is there really an o-s-d over the video within the browser window? If so, they most likely use something with OpenGL, yes.
google "webkit gtk vaapi gstreamer" there's a blog by the (pretty much world's only) person dealing with it.
and that means a bit of rewriting of both webkit and xulrunner so that they hand over the display bit to G2D. that's going to require some specialist work.
It would theoretically be possible to let libvdpau-sunxi use OpenGL/ES under the hood and blit everything into one surface/texture. But this maybe would drop the performance advantages compared to how we do know. And you depend on Mali....
... which would be bad. every person who's running with libre software at the moment would never be able to view videos.
There are some efforts to integrate/simulate the nv_gl_interop extension with libvdpau-sunxi and OpenGL/ES, which lets OpenGL use vdpau surface and vice versa. So it would be possible to let vdpau do the rendering and OpenGL/ES do the presentation. Problem is, that it's not finished and it's not intended by the spec anyway to either use VDPAU on ARM nor combine it with OpenGL/ES, as the nv_gl_interop only talks about i386 and OpenGL. <- This is the way kodi works with vdpau for example.
And the whole problem of this all is, that i currently cannot used with mainline kernel :)
i knoooow....
l.
Am 10.08.2016 um 16:30 schrieb Luke Kenneth Casson Leighton:
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
On Wed, Aug 10, 2016 at 8:23 AM, Andreas Baierl list@imkreisrum.de wrote:
Hi Luke,
Am 10.08.2016 um 00:16 schrieb Luke Kenneth Casson Leighton:
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
On Tue, Aug 9, 2016 at 10:40 PM, Russell Hyer russell.hyer@gmail.com wrote:
Hi Luke,
So, if I've understood this post correctly, video playback on vlc would perhaps be within the bounds of what the A20 hardware / software could do?
yes. ffplay, vlc, mpv, mplayer - all work fine, that's what this update was about: https://www.crowdsupply.com/eoma68/micro-desktop/updates/cedrus
basically by using the full capabilities of the A20's accelerated hardware (both CEDAR and G2D), the CPU usage is somewhere around the 10% mark. peanuts in other words.
the reason why those work is, through vdpau plugins there's two parts: "decode" part and "actual on-screen display" part. all those video players have an option to do *both* things, and the vdpau-sunxi plugin that the Cedrus team wrote goes "okay, we're configured to use G2D, let's just go for it".
that's why you end up with the overlay issues, btw, because the G2D hardware is told to write direct to an area on-screen, bypassing the X11 protocol.
Just a minor correction. The G2D (mixer processor) hardware (only on A10/A20) is only used within libvdpau-sunxi to blit one rgba surface into another one. It has nothing to do with bringing something *really* to screen, because it only handles the blit or fill actions on rgba surfaces. The only use of rgba surfaces that i know of, is to realize an OSD, subtitles etc... as an overlay over the video.
interesting.... because if i don't enable OSD i get absolutely nothing (black screen) on mpv and other players.
Arghh... You are the second one, that reports that issue :( Now i really have to investigate into this :p With enable you mean setting VDPAU_OSD=1 ? And I suppose you do this all in windows within X11?
But getting offtopic now. Maybe we should continue on irc...
rellla
i knoooow....
l.
arm-netbook mailing list arm-netbook@lists.phcomp.co.uk http://lists.phcomp.co.uk/mailman/listinfo/arm-netbook Send large attachments to arm-netbook@files.phcomp.co.uk
--- crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
On Wed, Aug 10, 2016 at 3:37 PM, Andreas Baierl list@imkreisrum.de wrote:
Arghh... You are the second one, that reports that issue :( Now i really have to investigate into this :p
:) using 16-bit 1280x720. i think.
With enable you mean setting VDPAU_OSD=1 ?
yes.
And I suppose you do this all in windows within X11?
yes.
But getting offtopic now. Maybe we should continue on irc...
sure.
El Wed, Aug 10, 2016 at 03:48:33PM +0100, Luke Kenneth Casson Leighton deia:
But getting offtopic now. Maybe we should continue on irc...
sure.
Continue wherever you want. But I don't think it is off-topic here.
Not sure about the list orginal intent, but these are issues that come up when you take certain arm SOCs and use them for a netbook, and look for free software support for the hardware so I'd say it is on topic. I assume most people here are interested in how to solve these issues, or what the status is while they are solved, or what kind of issues to expect in future projects...
At least for me it's been a very instructive discussion to read so far. IRC might be faster or more confortable for you, so fell free to use it but my vote is for welcoming you to talk about this here and thank you for teaching us. You don't get this kind of knowledge when you buy a wintel laptop.
--- crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
On Wed, Aug 10, 2016 at 4:02 PM, Xavi Drudis Ferran xdrudis@tinet.cat wrote:
El Wed, Aug 10, 2016 at 03:48:33PM +0100, Luke Kenneth Casson Leighton deia:
But getting offtopic now. Maybe we should continue on irc...
sure.
Continue wherever you want. But I don't think it is off-topic here.
Not sure about the list orginal intent, but these are issues that come up when you take certain arm SOCs and use them for a netbook, and look for free software support for the hardware so I'd say it is on topic. I assume most people here are interested in how to solve these issues, or what the status is while they are solved, or what kind of issues to expect in future projects...
At least for me it's been a very instructive discussion to read so far. IRC might be faster or more confortable for you, so fell free to use it but my vote is for welcoming you to talk about this here and thank you for teaching us. You don't get this kind of knowledge when you buy a wintel laptop.
good point. subject line even seems relevant which is amazing.
for firefox there is the addon Video Assistant that will auto play video from embedded & video sites in your chosen system video player. I find mpv works well for this because it has a very quick load time and a basic, minimal window that i then put on-top and have the video playing in the corner of my screen while i do other things.
--- crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
On Wed, Aug 10, 2016 at 2:35 AM, Alexander .S.T. Ross maillist_arm-netbook@aross.me wrote:
for firefox there is the addon Video Assistant that will auto play video from embedded & video sites in your chosen system video player.
ah ha! ok. i'll give it a shot.
El Wed, Aug 10, 2016 at 02:35:05AM +0100, Alexander .S.T. Ross deia:
for firefox there is the addon Video Assistant that will auto play video from embedded & video sites in your chosen system video player. I find mpv works well for this because it has a very quick load time and a basic, minimal window that i then put on-top and have the video playing in the corner of my screen while i do other things.
And maybe go to about:config in firefox and disable mediasource with
media.mediasource.enabled=false
mediasource (MSE) is a modern WebAPI used by vimeo, youtube and some popular video websites to throtle streams (or download stream chunks at different moments and resolutions/quality depending on instant network bandwith available) from javascript in the client. Besides cross-origin problems (I guess it's the implementation fault) and possible LibreJS issues, Video Asssitant can't get an URL to give the video player if you have mediasource enabled and go to websites that use mediasource. At least for now those sites degrade to http(s) urls if mediasource is unavailable in the browser.
Well, I just tried some examples, maybe it's not completely true generally, but it seems to work like this in my short experience.
On Wed, Aug 10, 2016 at 10:14:32AM +0200, Xavi Drudis Ferran wrote:
And maybe go to about:config in firefox and disable mediasource with
media.mediasource.enabled=false
If that turns out to be required: it is rather simple to create a package (at least on Debian) of a xul extension that merely sets several config directives. They will be applied the next time iceweasel / firefox is started.
I'm not sure how this works with the need for signing addons in recent versions of firefox, though.
On Tue, 9 Aug 2016 22:36:05 +0100 Luke Kenneth Casson Leighton lkcl@lkcl.net wrote:
.... ain't gonna work with the current code-base of either xulrunner (firefox and ice* types) or webkit (chrome, midori, qupzilla). darn.
the reason for webkit is: although gstreamer supports vdpau and vaapi, and even can be activated to do hardware-accelerated decode, webkit (at least as best i can find) the gtk+ variant then targets *opengl* as the output rendering engine, rather than asks VAAPI / VDPAU to handle it directly on the webkit engine's behalf.
Background: the website may need to do transformations on the data, read the data, or overlay several dozen layers of ads on top of the video, each with different clicky reactions.
Thus, a web browser outputting to a hw video layer would be non-compliant, as it would in most cases prevent those things (and especially for the ads, we can't have that, can we?).
So, you can't expect it to be mainlined in any popular browser, even if you develop such. It would break some sites, and block ads on others.
(plug: Fifth, my browser, is happily non-compliant, as it replaces all HTML5 video blocks with "download" and "stream" buttons, the stream button calling whatever video player you want. Similar to the FF extension mentioned ITT)
- Lauri
arm-netbook@lists.phcomp.co.uk
-
Alexander .S.T. Ross -
Andreas Baierl -
Lauri Kasanen -
Luke Kenneth Casson Leighton -
Russell Hyer -
Tzafrir Cohen -
Xavi Drudis Ferran