Re: [NTG-context] JavaScript in multimedia PDF

10 Aug 2021

      On Mon Aug 9, 2021 at 8:32 AM CEST, Pablo Rodriguez via ntg-context wrote:
...
On 8/8/21 11:00 PM, Michal Vlasák via ntg-context wrote:
...
Is there a newer document than "JavaScript for Acrobat API Reference"
(May 2015)?
Well, the link you provided
(https://mailman.ntg.nl/pipermail/ntg-context/2021/103023.html) is from
February 2021:
https://opensource.adobe.com/dc-acrobat-sdk-docs/acrobatsdk/pdfs/acrobatsdk_...
Here is the guide to developers (from the same date):
https://opensource.adobe.com/dc-acrobat-sdk-docs/acrobatsdk/pdfs/acrobatsdk_...
Thank you, I didn't realize that they updated the document after I
saved it. The (public) update also must have been more recently than
in February (I think the 2015 one was still up in May).

I wasn't every of the guide. It seems nice.
...
I get exactly the same results: screen object, but null rendition.
I have no idea what it is needed here.
Sorry, in the last mail I forgot to mention one idea I had. But now I
had time to try it, and in the end came up with three different ways to
get it working (at least the multimedia part, didn't test the rest).

It turns out, that Acrobat Reader ignores the names assigned to the
Renditions -- that's why it can't be found by name (this is not
complaint with the spec). But there is another mapping of Rendition
names and objects -- the "Renditions" name tree, that maps Unicode
encoded names of renditions to the respective objects. After generating
the name tree, the method "getRendition" works correctly. This is the
possibility number one -- generate the name tree.

Another possibility uses the fact, that the "openPlayer" method needs to
know which Rendition (what to play) and Screen annotation (where to
play) to target. While in ordinary JavaScript actions one needs to
specify them manually (because there is no context), in _Rendition_ PDF
actions (that can also execute JavaScript) there is a context -- the
action specifies the annotation and rendition at the PDF dictionary
level. So possibility number two is to use JavaScript Rendition actions.

Lastly, possibility number three. If one can't play the multimedia with
JavaScript, there still is a plain (no JavaScript) Rendition action,
that can do this -- "/OP 0" (StartRendering in ConTeXt). Also one can
chain several PDF actions one after the other. So if we first play the
Rendition through this ordinary Rendition action, then we can chain
arbitrary JavaScript after it. The order is important here, because the
second action doesn't start until first one is finished (which you may
or not desire, so feel free to swap the order).

All three possibilities work in Acrobat Reader 2021.005.20060. Depending
on the length of your audio, you may be interested in the fact, that
Acrobat Reader starts playing the audio from the start, if it didn't
finish it before. And also supports only embedded files.

Foxit Reader 11.0.1.49938 supports only the second and third way. It
also starts playing the same audio multiple times if started multiple
times (the same track plays multiple times over itself). All three file
types are supported, though I only check embedded and external files
(not URL files).

There are probably functions in the JS API to achieve better control and
consequently the same behaviour in both viewers, but I didn't look.

The first two ways are not currently possible in ConTeXt. The last one
should work, because ConTeXt should support chaining actions with
/Next, although I would have to check what is the user interface
(implementation is in lpdf.action).

I however, have a proof of concept in OpTeX, that uses the package from
my thesis (but I had to extend it for the Unicode encoded Rendition name
tree). To compile, you would need OpTeX and the updated "pdfextra.opm"
from
https://raw.githubusercontent.com/vlasakm/pdfextra/master/pdfextra.opm.
But the compiled PDF from the below example is attached.

With the proof of concept, and real world PDF document available, we
should probably get the functionality into ConTeXt. First way should
work if the name tree is added (I will look into that), but also needs a
better way to refer to the screen annotations, than what I am using
below (which maybe means more book keeping, have to check). The second
way would need a new executer and its shortcut, maybe something used
like this:

    \goto{...}[JSRendition{<rendition>, <javascript>}]

But I presume that more book keeping will be necessary.

The third way should hopefully already work.

Also, beware that both in my example, and in ConTeXt, multimedia
insertion produces whatsits, which especially when inserted with zero
dimensions like here for audio, may cause surprising interactions when
around normal typsetting content. Putting the whatsit insertion in a
page background is a way to keep it away. "back-swf.mkiv" suggests:

    \setupbackgrounds[page][background=resources]
    \setlayer[resources]{\placerenderingwindow[foo][foo]}

Michal Vlasák

% compile twice with "optex" and use pdfextra.opm from github
\fontfam[lm]
\load[pdfextra]
\hyperlinks\Blue\Blue

% The demo sound is randomly found, from Marianne Gagnon
% https://soundbible.com/1682-Robot-Blip.html

% initialization javascript (runs on document open)
\dljavascript[whatever]{%
function play_rendition(label) {
    this.pageNum = 0;
    var rendition = this.media.getRendition(label);
    var screen = this.media.getAnnots({ nPage: 0 })[0];
    console.println(label);
    console.println(rendition);
    console.println(screen);
    console.show();
    var player = app.media.openPlayer({
        rendition: rendition,
        annot: screen,
    });
}
}

% Rendition + Screen annotation (beware of the whatsit node!)
\newbox \backgroundbox
\setbox\backgroundbox=\hbox{\render[sound.mp3]{}}
\pgbackground={\box\backgroundbox}

% First possibility
\hlink[js:play_rendition("sound.mp3");]
  {Play sound with ordinary JavaScript action
   (function \code{GoToFirstSlide})}

% Second possibility
\sdef{_pdfextra_renditionaction:jsplay}{/JS (
    % argsDWIM ("Do what I mean") is what openPlayer
    % calls internally on its argument
    console.println(app.media.argsDWIM({}));
    console.show();
    var player = app.media.openPlayer({});
)}
\hlink[rendition::jsplay]
  {Play sound with (JavaScript) Rendition action (\code{rendition}
   and \code{annot} inherited from the action)}

% Third possibility
\hlink
  [rendition::play,
   js:app.alert("javascript code");]
  {Play sound with (Play) Rendition action chained before
   JavaScript action}

\bye