JavaScript in multimedia PDF
Dear list, I have the following sample: \setupinteraction [state=start] \startJSpreamble varia used now this.pageNum = 0 ; // start at 0 function GoToFirstSlide(label) { this.pageNum = 0 ; var rendition = this.media.getRendition(label) ; var player = app.media.openPlayer({ rendition: rendition, }); } function GoToLastSlide(label) { this.pageNum = this.numPages ; } function GoToNextSlide(label) { ++this.pageNum ; } function GoToPreviousSlide(label) { --this.pageNum ; } function SwitchFS() { if (app.fs.isFullScreen == true) app.fs.isFullScreen = false ; else app.fs.isFullScreen = true ; } \stopJSpreamble \setuppapersize[A9, landscape] \definefontfamily [mainface] [rm] [Latin Modern Sans] \definefontfamily [mainface] [ss] [Hans] \setupbodyfont [mainface, 25pt] \setupalign[middle] \setupfooter [style={\ss}] \def\SlideNavigationButtons{% \goto{A}[JS(GoToFirstSlide{mainsound})]% \goto{K}[JS(GoToPreviousSlide{mainsound})]% \goto{L}[JS(GoToNextSlide{mainsound})]% \goto{B}[JS(GoToLastSlide{mainsound})] \goto{{\ssb Q}}[JS(SwitchFS{})]% } \setupfootertexts[\SlideNavigationButtons] \starttext \definerenderingwindow[soundplace] [width=0pt, height=0pt] \useexternalrendering[mainsound][audio/mp3][sound.mp3][embed=yes] \placerenderingwindow[soundplace][mainsound] \dorecurse{25}{\null\page} \stoptext I don’t know why I get the following message: TypeError: a.doc is undefined It seems to be caused by "app.media.openPlayer", but the code is copied from the documentation Adobe released early this year. Does anyone know what is wrong here or what am I missing? Many thanks for your help in advance, Pablo -- http://www.ousia.tk
On Sun Aug 8, 2021 at 8:22 PM CEST, Pablo Rodriguez via ntg-context wrote:
I don’t know why I get the following message:
TypeError: a.doc is undefined
It seems to be caused by "app.media.openPlayer", but the code is copied from the documentation Adobe released early this year.
Is there a newer document than "JavaScript for Acrobat API Reference" (May 2015)? Anyways, I thought that the culprit is that you only specify the rendition (what to play) and not screen (where to play): function GoToFirstSlide(label) { this.pageNum = 0 ; var rendition = this.media.getRendition(label) ; +var screen = this.media.getAnnots({ nPage: 0})[0]; var player = app.media.openPlayer({ rendition: rendition, + annot: screen, }); } But as it turns out, while the screen query succesfully returns an object, the rendition query returns null. The PDF 1.7 standard specifies that the name (/N) of Rendition PDF object should be Unicode, but that doesn't seem to make it work either, as does deleting the name of the media clip (which was the same). (But I now get "Invalid arguments" error for the "app.media.openPlayer" call, which is kind of expected.) Can you confirm, that "rendition" is null on your side as well? console.println(rendition); console.println(screen); console.show(); Michal Vlasák
On 8/8/21 11:00 PM, Michal Vlasák via ntg-context wrote:
On Sun Aug 8, 2021 at 8:22 PM CEST, Pablo Rodriguez via ntg-context wrote:
I don’t know why I get the following message:
TypeError: a.doc is undefined
It seems to be caused by "app.media.openPlayer", but the code is copied from the documentation Adobe released early this year.
Is there a newer document than "JavaScript for Acrobat API Reference" (May 2015)?
Well, the link you provided (https://mailman.ntg.nl/pipermail/ntg-context/2021/103023.html) is from February 2021: https://opensource.adobe.com/dc-acrobat-sdk-docs/acrobatsdk/pdfs/acrobatsdk_... Here is the guide to developers (from the same date): https://opensource.adobe.com/dc-acrobat-sdk-docs/acrobatsdk/pdfs/acrobatsdk_...
Anyways, I thought that the culprit is that you only specify the rendition (what to play) and not screen (where to play):
function GoToFirstSlide(label) { this.pageNum = 0 ; var rendition = this.media.getRendition(label) ; +var screen = this.media.getAnnots({ nPage: 0})[0]; var player = app.media.openPlayer({ rendition: rendition, + annot: screen, }); }
But as it turns out, while the screen query succesfully returns an object, the rendition query returns null.
The PDF 1.7 standard specifies that the name (/N) of Rendition PDF object should be Unicode, but that doesn't seem to make it work either, as does deleting the name of the media clip (which was the same).
(But I now get "Invalid arguments" error for the "app.media.openPlayer" call, which is kind of expected.)
Can you confirm, that "rendition" is null on your side as well?
console.println(rendition); console.println(screen); console.show();
I get exactly the same results: screen object, but null rendition. I have no idea what it is needed here. Many thanks for your help, Pablo -- http://www.ousia.tk
On Mon Aug 9, 2021 at 8:32 AM CEST, Pablo Rodriguez via ntg-context wrote:
On 8/8/21 11:00 PM, Michal Vlasák via ntg-context wrote:
Is there a newer document than "JavaScript for Acrobat API Reference" (May 2015)?
Well, the link you provided (https://mailman.ntg.nl/pipermail/ntg-context/2021/103023.html) is from February 2021:
https://opensource.adobe.com/dc-acrobat-sdk-docs/acrobatsdk/pdfs/acrobatsdk_...
Here is the guide to developers (from the same date):
https://opensource.adobe.com/dc-acrobat-sdk-docs/acrobatsdk/pdfs/acrobatsdk_...
Thank you, I didn't realize that they updated the document after I saved it. The (public) update also must have been more recently than in February (I think the 2015 one was still up in May). I wasn't every of the guide. It seems nice.
I get exactly the same results: screen object, but null rendition.
I have no idea what it is needed here.
Sorry, in the last mail I forgot to mention one idea I had. But now I had time to try it, and in the end came up with three different ways to get it working (at least the multimedia part, didn't test the rest). It turns out, that Acrobat Reader ignores the names assigned to the Renditions -- that's why it can't be found by name (this is not complaint with the spec). But there is another mapping of Rendition names and objects -- the "Renditions" name tree, that maps Unicode encoded names of renditions to the respective objects. After generating the name tree, the method "getRendition" works correctly. This is the possibility number one -- generate the name tree. Another possibility uses the fact, that the "openPlayer" method needs to know which Rendition (what to play) and Screen annotation (where to play) to target. While in ordinary JavaScript actions one needs to specify them manually (because there is no context), in _Rendition_ PDF actions (that can also execute JavaScript) there is a context -- the action specifies the annotation and rendition at the PDF dictionary level. So possibility number two is to use JavaScript Rendition actions. Lastly, possibility number three. If one can't play the multimedia with JavaScript, there still is a plain (no JavaScript) Rendition action, that can do this -- "/OP 0" (StartRendering in ConTeXt). Also one can chain several PDF actions one after the other. So if we first play the Rendition through this ordinary Rendition action, then we can chain arbitrary JavaScript after it. The order is important here, because the second action doesn't start until first one is finished (which you may or not desire, so feel free to swap the order). All three possibilities work in Acrobat Reader 2021.005.20060. Depending on the length of your audio, you may be interested in the fact, that Acrobat Reader starts playing the audio from the start, if it didn't finish it before. And also supports only embedded files. Foxit Reader 11.0.1.49938 supports only the second and third way. It also starts playing the same audio multiple times if started multiple times (the same track plays multiple times over itself). All three file types are supported, though I only check embedded and external files (not URL files). There are probably functions in the JS API to achieve better control and consequently the same behaviour in both viewers, but I didn't look. The first two ways are not currently possible in ConTeXt. The last one should work, because ConTeXt should support chaining actions with /Next, although I would have to check what is the user interface (implementation is in lpdf.action). I however, have a proof of concept in OpTeX, that uses the package from my thesis (but I had to extend it for the Unicode encoded Rendition name tree). To compile, you would need OpTeX and the updated "pdfextra.opm" from https://raw.githubusercontent.com/vlasakm/pdfextra/master/pdfextra.opm. But the compiled PDF from the below example is attached. With the proof of concept, and real world PDF document available, we should probably get the functionality into ConTeXt. First way should work if the name tree is added (I will look into that), but also needs a better way to refer to the screen annotations, than what I am using below (which maybe means more book keeping, have to check). The second way would need a new executer and its shortcut, maybe something used like this: \goto{...}[JSRendition{<rendition>, <javascript>}] But I presume that more book keeping will be necessary. The third way should hopefully already work. Also, beware that both in my example, and in ConTeXt, multimedia insertion produces whatsits, which especially when inserted with zero dimensions like here for audio, may cause surprising interactions when around normal typsetting content. Putting the whatsit insertion in a page background is a way to keep it away. "back-swf.mkiv" suggests: \setupbackgrounds[page][background=resources] \setlayer[resources]{\placerenderingwindow[foo][foo]} Michal Vlasák % compile twice with "optex" and use pdfextra.opm from github \fontfam[lm] \load[pdfextra] \hyperlinks\Blue\Blue % The demo sound is randomly found, from Marianne Gagnon % https://soundbible.com/1682-Robot-Blip.html % initialization javascript (runs on document open) \dljavascript[whatever]{% function play_rendition(label) { this.pageNum = 0; var rendition = this.media.getRendition(label); var screen = this.media.getAnnots({ nPage: 0 })[0]; console.println(label); console.println(rendition); console.println(screen); console.show(); var player = app.media.openPlayer({ rendition: rendition, annot: screen, }); } } % Rendition + Screen annotation (beware of the whatsit node!) \newbox \backgroundbox \setbox\backgroundbox=\hbox{\render[sound.mp3]{}} \pgbackground={\box\backgroundbox} % First possibility \hlink[js:play_rendition("sound.mp3");] {Play sound with ordinary JavaScript action (function \code{GoToFirstSlide})} % Second possibility \sdef{_pdfextra_renditionaction:jsplay}{/JS ( % argsDWIM ("Do what I mean") is what openPlayer % calls internally on its argument console.println(app.media.argsDWIM({})); console.show(); var player = app.media.openPlayer({}); )} \hlink[rendition::jsplay] {Play sound with (JavaScript) Rendition action (\code{rendition} and \code{annot} inherited from the action)} % Third possibility \hlink [rendition::play, js:app.alert("javascript code");] {Play sound with (Play) Rendition action chained before JavaScript action} \bye
On 8/10/21 11:52 PM, Michal Vlasák via ntg-context wrote:
On Mon Aug 9, 2021 at 8:32 AM CEST, Pablo Rodriguez via ntg-context wrote: [...]
I get exactly the same results: screen object, but null rendition.
I have no idea what it is needed here.
Sorry, in the last mail I forgot to mention one idea I had. But now I had time to try it, and in the end came up with three different ways to get it working (at least the multimedia part, didn't test the rest).
Many thanks for your in-depth reply and explanation, Michal. Since there are some related issues, I think it is better to discuss it in private. Many thanks for your help again, Pablo -- http://www.ousia.tk
More context is in previous messages, but here is the patch for ConTeXt to make scripting multimedia possible: --- a/tex/context/base/mkxl/lpdf-wid.lmt +++ b/tex/context/base/mkxl/lpdf-wid.lmt @@ -649,6 +649,7 @@ local function insertrenderingwindow(specification) Subtype = pdfconstant("Screen"), P = pdfreference(pdfpagereference(page)), A = a, -- needed in order to make the annotation clickable (i.e. don't bark) + T = pdfunicode(label), -- title (for JS) Border = bs, C = bc, AA = actions, @@ -693,7 +694,7 @@ local function insertrendering(specification) -- } -- } local parameters = pdfdictionary { - Type = pdfconstant(MediaPermissions), + Type = pdfconstant("MediaPermissions"), TF = pdfstring("TEMPALWAYS"), -- TEMPNEVER TEMPEXTRACT TEMPACCESS TEMPALWAYS / needed for acrobat/wmp } local descriptor = pdfdictionary { @@ -723,7 +724,7 @@ local function insertrendering(specification) local rendition = pdfdictionary { Type = pdfconstant("Rendition"), S = pdfconstant("MR"), - N = label, + N = pdfunicode(label), C = pdfreference(pdfflushobject(clip)), } mf[label] = pdfreference(pdfflushobject(rendition)) @@ -761,6 +762,20 @@ function codeinjections.processrendering(label) end end +-- needed mapping for access from JS +local function flushrenderings() + if next(mf) then + local r = pdfarray() + for label, reference in sortedhash(mf) do + r[#r+1] = pdfunicode(label) + r[#r+1] = reference -- already a reference + end + lpdf.addtonames("Renditions",pdfreference(pdfflushobject(pdfdictionary{ Names = r }))) + end +end + +lpdf.registerdocumentfinalizer(flushrenderings,"renderings") + function codeinjections.insertrenderingwindow(specification) local label = specification.label codeinjections.processrendering(label) The patch is also be available here https://github.com/vlasakm/context-mirror/commit/99f81beae0d13f1aecc20be187a... until it is applied (for full file download, which is available under the three dots next to the file name). The created Screen annotation will be given the title, that corresponds to the second \placerenderingwindow parameter. So here: \definerenderingwindow[soundplace] [width=0pt, height=0pt] \useexternalrendering[mainsound][audio/mp3][sound.mp3][embed=yes] \placerenderingwindow[soundplace][mainsound] both the screen annotation and rendition are available under the name "mainsound". The name "soundplace" can be anything (even "mainsound"), it doesn't propagate to the PDF file in any way. And here is the patch required to make Pablo's example work: --- a/pablo.tex +++ b/pablo.tex @@ -7,8 +7,10 @@ function GoToFirstSlide(label) { this.pageNum = 0 ; var rendition = this.media.getRendition(label) ; +var screen = this.media.getAnnot({nPage: 0, cAnnotTitle: label}) ; var player = app.media.openPlayer({ rendition: rendition, + annot: screen, }); } The second possibility I proposed previously is not as general, and implementing it is not worth, until it is really needed (which I presume won't happen). Michal Vlasák
Hi
- Type = pdfconstant(MediaPermissions), + Type = pdfconstant("MediaPermissions"),
That's an interesting one ... easilly goes unnoticed ... thanks Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On Thu Aug 12, 2021 at 12:55 AM CEST, Hans Hagen via ntg-context wrote:
Hi
- Type = pdfconstant(MediaPermissions), + Type = pdfconstant("MediaPermissions"),
That's an interesting one ... easilly goes unnoticed ... thanks
To put credit where its due, this was Pablo's find. Indeed very subtle, I didn't notice at all, and PDF viewers also probably mostly ignore these "/Type"s. Michal
On 8/12/21 3:02 AM, Michal Vlasák via ntg-context wrote:
On Thu Aug 12, 2021 at 12:55 AM CEST, Hans Hagen via ntg-context wrote:
Hi
- Type = pdfconstant(MediaPermissions), + Type = pdfconstant("MediaPermissions"),
That's an interesting one ... easilly goes unnoticed ... thanks
To put credit where its due, this was Pablo's find.
Well, I only noticed that in the generated PDF code read: /Type /none It was Michal who found the bug in the Lua code. Many thanks for the work improving multimedia in ConTeXt, Pablo -- http://www.ousia.tk
participants (3)
-
Hans Hagen
-
Michal Vlasák
-
Pablo Rodriguez