I'm wondering if there has been a regression in the rendering of CJK recently? I'm attaching two screenshots from the same code: one with lmtx-20210509 (correct); and the other lmtx-20210613 (incomplete). This is what I have in the setups: % Support for Japanese \setscript[hanzi] \starttypescript[serif][sourcehanserifregular] \definefontsynonym[Serif][name:sourcehanserifregular][features={aalt,h ist,kern,liga}] \stoptypescript \definetypeface[sourcehanserifregular][rm][serif][sourcehanserifregula r] \define[1]\JA{\language[ja]{{\setscript[hanzi]\sourcehanserifregular #1}}} And this is the code behind the screenshots: {\tfx \everypar{\hangafter=1\hangindent=3em\relax} Karashima, N.~(1977) {\it Indo nyūmon} = \JA{\tfx インド入門} = {\it Introducing India}. Tōkyō: Tōkyō Daigaku Shuppankai. \useURL[url1][https://ci.nii.ac.jp/ncid/BN00345890][][\hyphenatedurl{https://ci.nii.ac.jp/ncid/BN00345890}]\from[url1 ] \hl[2.5] (1980) {\it Indasu bunmei: indo bunka no genryū o nasumono} = \JA{\tfx インダス文明 : インド文化の源流をなすもの} = {\it The Indus Civilization}. NHK bukkusu, 375. Tōkyō: NHK bukkusu. \useURL[url2][https://ci.nii.ac.jp/ncid/BN00354483][][\hyphenatedurl{https://ci.nii.ac.jp/ncid/BN00354483}]\from[url2 ] \hl[2.5] (1985) {\it Indo sekai no rekishizō} = \JA{\tfx インド世界の歴史像} = {\it Historical Image of the Indian World}. Minzoku no sekaishi, 7. Tōkyō: Yamakawa Shuppansha. \useURL[url3][https://ci.nii.ac.jp/ncid/BN00332312][][\hyphenatedurl{https://ci.nii.ac.jp/ncid/BN00332312}]\from[url3 ] \hl[2.5] (1992) {\it Minamiajia o shiru jiten} = \JA{\tfx 南アジアを知る事典} = {\it Cyclopedia of South Asia: indo suriranka nepāru pakisutan banguradishu būtan morudibu}. \JA{\tfx 東京 : 平凡社}. \useURL[url4][https://ci.nii.ac.jp/ncid/BN08142729][][\hyphenatedurl{https://ci.nii.ac.jp/ncid/BN08142729}]\from[url4 ] ... } Best, Richard -- T +6433121699 M +64210640216 E rmahoney@indica-et-buddhica.org IM https://t.me/rmahoney W https://indica-et-buddhica.org/ Indica et Buddhica Littledene Bay Road Oxford NZ
On 6/13/2021 9:42 PM, Richard Mahoney wrote:
I'm wondering if there has been a regression in the rendering of CJK recently? I'm attaching two screenshots from the same code: one with lmtx-20210509 (correct); and the other lmtx-20210613 (incomplete). no, it's a side effect of something else (which shows up with these extremeely large fonts > 65k glyphs) .. i'll look into it tomorrow
----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On 6/13/2021 9:42 PM, Richard Mahoney wrote:
I'm wondering if there has been a regression in the rendering of CJK recently? I'm attaching two screenshots from the same code: one with lmtx-20210509 (correct); and the other lmtx-20210613 (incomplete). no, it's a side effect of something else (which shows up with these extremeely large fonts > 65k glyphs) .. i'll look into it tomorrow I uploaded a new lmtx ... kind of experimental because I changes some of
On 6/13/2021 10:54 PM, Hans Hagen wrote: the background bist that deals with embedding. The problem is that we need to deal with the previously reported clash between different unicode entries that share shapes (normally no issue but in this case it was a side effect of 'effective' monospaces where the font decided that invisible shapes should he visual anyway) as well as with the fact that soem cjk fonts have many duplicates which makes us cross the 65 boundary. The variant approach is ok but I might have overlooked some soecial cases. Because this also drops 'stream compatibility' between mkiv and lmtx (which had already become somewhat loose) I can now also clean up (simplify) some other parts of the font system but let's do that stepwise (I'm in no hurry here). Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On 6/13/2021 9:42 PM, Richard Mahoney wrote:
I'm wondering if there has been a regression in the rendering of CJK recently? I'm attaching two screenshots from the same code: one with lmtx-20210509 (correct); and the other lmtx-20210613 (incomplete). no, it's a side effect of something else (which shows up with these extremeely large fonts > 65k glyphs) .. i'll look into it tomorrow I uploaded a new lmtx ... kind of experimental because I changes some of
Thank you very much for this Hans. And especially for resolving
what looks like quite a complex issue in next to no time.
As it happens I've found LMTX just fine for "production", so long as
I keep copies of a few versions that I know are working well.
I settled on SciTE a long time ago and tend to keep a few start
up scripts that make reverting to a previous version simple:
-r-x------ 1 rbm49 rbm49 210 Dec 20 16:54 SciTE-start-lmtx-20200920.sh
-r-x------ 1 rbm49 rbm49 210 Jun 13 22:04 SciTE-start-lmtx-20210509.sh
-rwx------ 1 rbm49 rbm49 210 Jun 15 08:50 SciTE-start-lmtx-latest.sh
#!/bin/bash
#
# SciTE-start-lmtx-20200920.sh
PATH="${HOME}/lmtx-20200920/tex/texmf-linux-64/bin:${HOME}/lmtx-
20200920/bin:${HOME}/bin:${HOME}/.local/bin:/opt/VirtualBox:${PATH}"
cd /home/rbm49 &&
exec /usr/local/bin/SciTE && exit 0
exit 2
But really, taking things back a version or two isn't needed all that
often
as issues tend to be sorted out quite quickly with all the testing
that people
are doing -- so thanks for that too. :)
Best, Richard
--
T +6433121699 M +64210640216 E rmahoney@indica-et-buddhica.org
IM https://t.me/rmahoney W https://indica-et-buddhica.org/
Indica et Buddhica Littledene Bay Road Oxford NZ
-----Original Message-----
From: Hans Hagen
On 6/15/2021 12:21 AM, Richard Mahoney wrote:
Thank you very much for this Hans. And especially for resolving what looks like quite a complex issue in next to no time.
As it happens I've found LMTX just fine for "production", so long as I keep copies of a few versions that I know are working well.
I settled on SciTE a long time ago and tend to keep a few start up scripts that make reverting to a previous version simple:
-r-x------ 1 rbm49 rbm49 210 Dec 20 16:54 SciTE-start-lmtx-20200920.sh -r-x------ 1 rbm49 rbm49 210 Jun 13 22:04 SciTE-start-lmtx-20210509.sh -rwx------ 1 rbm49 rbm49 210 Jun 15 08:50 SciTE-start-lmtx-latest.sh
btw, When testing your example I noticed that sumatra pdf (which i use as viewer) doesn't handle selecting the cjk text well. I looked at the source (on github) and get the impression that it might be some font bbox issue (these asian fonts have hugee bboxes); (also old) acrobat and browsers do fine as does an old windows okular I have installed. Does anyone know of a minimalistic mupdf based pdf viewer that has a decent gui but not all this (semi) epub, xps, reflow, etc stuff on board? Just the mupdf pdf core.
#!/bin/bash # # SciTE-start-lmtx-20200920.sh PATH="${HOME}/lmtx-20200920/tex/texmf-linux-64/bin:${HOME}/lmtx-20200920/bin:${HOME}/bin:${HOME}/.local/bin:/opt/VirtualBox:${PATH}" cd /home/rbm49 && exec /usr/local/bin/SciTE && exit 0 exit 2
indeed, my collegue does something like that (per project actually) and updates when things come out the same (or better or when some new feature is needed)
But really, taking things back a version or two isn't needed all that often as issues tend to be sorted out quite quickly with all the testing that people are doing -- so thanks for that too. :)
right, and thanks to that rapid feedback we were able to advance and cleanup the engine quite a bit as well as apply new features to context; without users that test and apply and demand new features there wouild be no development thanks, Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On Tue, Jun 15, 2021 at 9:13 AM Hans Hagen
On 6/15/2021 12:21 AM, Richard Mahoney wrote:
Thank you very much for this Hans. And especially for resolving what looks like quite a complex issue in next to no time.
As it happens I've found LMTX just fine for "production", so long as I keep copies of a few versions that I know are working well.
I settled on SciTE a long time ago and tend to keep a few start up scripts that make reverting to a previous version simple:
-r-x------ 1 rbm49 rbm49 210 Dec 20 16:54 SciTE-start-lmtx-20200920.sh -r-x------ 1 rbm49 rbm49 210 Jun 13 22:04 SciTE-start-lmtx-20210509.sh -rwx------ 1 rbm49 rbm49 210 Jun 15 08:50 SciTE-start-lmtx-latest.sh
btw, When testing your example I noticed that sumatra pdf (which i use as viewer) doesn't handle selecting the cjk text well. I looked at the source (on github) and get the impression that it might be some font bbox issue (these asian fonts have hugee bboxes); (also old) acrobat and browsers do fine as does an old windows okular I have installed.
Does anyone know of a minimalistic mupdf based pdf viewer that has a decent gui but not all this (semi) epub, xps, reflow, etc stuff on board? Just the mupdf pdf core.
perhaps the mupdf app for android. Another one is evince for windows: https://portableapps.com/apps/office/evince_portable -- luigi
On 6/15/2021 10:25 AM, luigi scarso wrote:
On Tue, Jun 15, 2021 at 9:13 AM Hans Hagen
mailto:j.hagen@xs4all.nl> wrote: On 6/15/2021 12:21 AM, Richard Mahoney wrote: > Thank you very much for this Hans. And especially for resolving > what looks like quite a complex issue in next to no time. > > As it happens I've found LMTX just fine for "production", so long as > I keep copies of a few versions that I know are working well. > > I settled on SciTE a long time ago and tend to keep a few start > up scripts that make reverting to a previous version simple: > > -r-x------ 1 rbm49 rbm49 210 Dec 20 16:54 SciTE-start-lmtx-20200920.sh > -r-x------ 1 rbm49 rbm49 210 Jun 13 22:04 SciTE-start-lmtx-20210509.sh > -rwx------ 1 rbm49 rbm49 210 Jun 15 08:50 SciTE-start-lmtx-latest.sh
btw, When testing your example I noticed that sumatra pdf (which i use as viewer) doesn't handle selecting the cjk text well. I looked at the source (on github) and get the impression that it might be some font bbox issue (these asian fonts have hugee bboxes); (also old) acrobat and browsers do fine as does an old windows okular I have installed.
Does anyone know of a minimalistic mupdf based pdf viewer that has a decent gui but not all this (semi) epub, xps, reflow, etc stuff on board? Just the mupdf pdf core.
perhaps the mupdf app for android.
the windows mupdf viewer is somewhat crippled (no selection) but searching for something shows the right bbox
Another one is evince for windows: https://portableapps.com/apps/office/evince_portable https://portableapps.com/apps/office/evince_portable I tried it and it does the selection ok (but it's some 60MB so not lean and mean).
I also tried xpdfreader (some 30B) which also selects ok. Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On Tue, 15 Jun 2021 13:07:04 +0200
Hans Hagen
the windows mupdf viewer is somewhat crippled (no selection) but searching for something shows the right bbox
I do not know about Windows ports, but mupdf and mupdf-gl differ, not only in their rendering but also in their keystrokes and mouse bindings. Alan
On 6/15/2021 7:53 PM, Alan Braslau wrote:
On Tue, 15 Jun 2021 13:07:04 +0200 Hans Hagen
wrote: the windows mupdf viewer is somewhat crippled (no selection) but searching for something shows the right bbox
I do not know about Windows ports, but mupdf and mupdf-gl differ, not only in their rendering but also in their keystrokes and mouse bindings. yes there are windows versions, but just as rudimentary and no text selection (copy)
----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On Tue, Jun 15, 2021 at 8:29 PM Hans Hagen
On 6/15/2021 7:53 PM, Alan Braslau wrote:
On Tue, 15 Jun 2021 13:07:04 +0200 Hans Hagen
wrote: the windows mupdf viewer is somewhat crippled (no selection) but searching for something shows the right bbox
I do not know about Windows ports, but mupdf and mupdf-gl differ, not only in their rendering but also in their keystrokes and mouse bindings.
mupdf-gl (the advanced one) uses opengl, but unfortunately it breaks when the drivers are updated; mupdf uses x11, and it's much more stable. To be honest, under ubuntu okular and evince are still better than mupdf. -- luigi
On 6/15/2021 8:47 PM, luigi scarso wrote:
On Tue, Jun 15, 2021 at 8:29 PM Hans Hagen
mailto:j.hagen@xs4all.nl> wrote: On 6/15/2021 7:53 PM, Alan Braslau wrote: > On Tue, 15 Jun 2021 13:07:04 +0200 > Hans Hagen
mailto:j.hagen@xs4all.nl> wrote: > >> the windows mupdf viewer is somewhat crippled (no selection) but >> searching for something shows the right bbox > > I do not know about Windows ports, but mupdf and mupdf-gl differ, not > only in their rendering but also in their keystrokes and mouse bindings. mupdf-gl (the advanced one) uses opengl, but unfortunately it breaks when the drivers are updated; mupdf uses x11, and it's much more stable. To be honest, under ubuntu okular and evince are still better than mupdf.
Did you try this one? http://www.xpdfreader.com/download.html (Not sure if we have a wiki page for viewers that keeps track of the quality.) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On Tue, Jun 15, 2021 at 9:00 PM Hans Hagen
On 6/15/2021 8:47 PM, luigi scarso wrote:
On Tue, Jun 15, 2021 at 8:29 PM Hans Hagen
mailto:j.hagen@xs4all.nl> wrote: On 6/15/2021 7:53 PM, Alan Braslau wrote: > On Tue, 15 Jun 2021 13:07:04 +0200 > Hans Hagen
mailto:j.hagen@xs4all.nl> wrote: > >> the windows mupdf viewer is somewhat crippled (no selection) but >> searching for something shows the right bbox > > I do not know about Windows ports, but mupdf and mupdf-gl differ, not
> only in their rendering but also in their keystrokes and mouse bindings.
mupdf-gl (the advanced one) uses opengl, but unfortunately it breaks when the drivers are updated; mupdf uses x11, and it's much more stable. To be honest, under ubuntu okular and evince are still better than mupdf.
Did you try this one?
I doesn't workunder ubuntu 20.04 libicudata.so.56 => not found I have libicudata.so.66 -- luigi
On Wed, 16 Jun 2021 17:14:22 +0200
luigi scarso
Did you try this one?
I doesn't workunder ubuntu 20.04 libicudata.so.56 => not found I have libicudata.so.66
The ubuntu package must be built incorrectly, for I have /usr/local/lib/libicudata.so.69 and xpdf works fine (freebsd). Alan
On Wed, Jun 16, 2021 at 10:53 PM Alan Braslau
On Wed, 16 Jun 2021 17:14:22 +0200 luigi scarso
wrote: Did you try this one?
I doesn't workunder ubuntu 20.04 libicudata.so.56 => not found I have libicudata.so.66
The ubuntu package must be built incorrectly, for I have /usr/local/lib/libicudata.so.69 and xpdf works fine (freebsd).
Alan
Hm, I think you have libicui18n.so.56 installed somewhere or your copy is using the "69" version. $> export LD_DEBUG=all; ./xpdf &>out ; unset LD_DEBUG says 1525647: file=libicui18n.so.56 [0]; needed by /tmp/xpdf/xpdf/XpdfReader-linux64-4.03/lib/libQt5Core.so.5 [0] 1525647: find library=libicui18n.so.56 [0]; searching : :<several paths > : 1525647: ./xpdf.bin: error while loading shared libraries: libicui18n.so.56: cannot open shared object file: No such file or directory -- luigi
On Thu, Jun 17, 2021 at 11:33 AM luigi scarso
On Wed, Jun 16, 2021 at 10:53 PM Alan Braslau
wrote: On Wed, 16 Jun 2021 17:14:22 +0200 luigi scarso
wrote: Did you try this one?
I doesn't workunder ubuntu 20.04 libicudata.so.56 => not found I have libicudata.so.66
The ubuntu package must be built incorrectly, for I have /usr/local/lib/libicudata.so.69 and xpdf works fine (freebsd).
Alan
Hm, I think you have libicui18n.so.56 installed somewhere or your copy is using the "69" version.
$> export LD_DEBUG=all; ./xpdf &>out ; unset LD_DEBUG says 1525647: file=libicui18n.so.56 [0]; needed by /tmp/xpdf/xpdf/XpdfReader-linux64-4.03/lib/libQt5Core.so.5 [0] 1525647: find library=libicui18n.so.56 [0]; searching : :<several paths > : 1525647: ./xpdf.bin: error while loading shared libraries: libicui18n.so.56: cannot open shared object file: No such file or directory
OK, in my case it seems that I don't need the libQt*so under lib/. So I have renamed it as _lib, and now xpdf runs ok. -- luigi
On Tue, 15 Jun 2021 20:29:26 +0200
Hans Hagen
On 6/15/2021 7:53 PM, Alan Braslau wrote:
On Tue, 15 Jun 2021 13:07:04 +0200 Hans Hagen
wrote: the windows mupdf viewer is somewhat crippled (no selection) but searching for something shows the right bbox
I do not know about Windows ports, but mupdf and mupdf-gl differ, not only in their rendering but also in their keystrokes and mouse bindings. yes there are windows versions, but just as rudimentary and no text selection (copy)
"The right mouse button selects a region and copies the marked text to the clipboard." Maybe this is broken (as Luigi suggests) or requires some special setup/configuration? Alan
On 6/15/2021 8:54 PM, Alan Braslau wrote:
On Tue, 15 Jun 2021 20:29:26 +0200 Hans Hagen
wrote: On 6/15/2021 7:53 PM, Alan Braslau wrote:
On Tue, 15 Jun 2021 13:07:04 +0200 Hans Hagen
wrote: the windows mupdf viewer is somewhat crippled (no selection) but searching for something shows the right bbox
I do not know about Windows ports, but mupdf and mupdf-gl differ, not only in their rendering but also in their keystrokes and mouse bindings. yes there are windows versions, but just as rudimentary and no text selection (copy)
"The right mouse button selects a region and copies the marked text to the clipboard."
Maybe this is broken (as Luigi suggests) or requires some special setup/configuration? I was refereing to a textual copy, not an image one. So one selects 'abc' and gets 'abc' (with the right unicodes).
Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On Tue, 15 Jun 2021 20:57:09 +0200
Hans Hagen
On 6/15/2021 8:54 PM, Alan Braslau wrote:
On Tue, 15 Jun 2021 20:29:26 +0200 Hans Hagen
wrote: On 6/15/2021 7:53 PM, Alan Braslau wrote:
On Tue, 15 Jun 2021 13:07:04 +0200 Hans Hagen
wrote: the windows mupdf viewer is somewhat crippled (no selection) but searching for something shows the right bbox
I do not know about Windows ports, but mupdf and mupdf-gl differ, not only in their rendering but also in their keystrokes and mouse bindings. yes there are windows versions, but just as rudimentary and no text selection (copy)
"The right mouse button selects a region and copies the marked text to the clipboard."
Maybe this is broken (as Luigi suggests) or requires some special setup/configuration? I was refereing to a textual copy, not an image one. So one selects 'abc' and gets 'abc' (with the right unicodes).
That is what I get using mupdf (x11). I found this comment: "Copy-paste will only work if mupdf-gl is built against the bundled copy of FreeGLUT, which adds the functions glutSetClipboard and glutGetClipboard." And, for example: https://github.com/NixOS/nixpkgs/issues/104017
Richard Mahoney schrieb am 13.06.2021 um 21:42:
I'm wondering if there has been a regression in the rendering of CJK recently? I'm attaching two screenshots from the same code: one with lmtx-20210509 (correct); and the other lmtx-20210613 (incomplete).
This is what I have in the setups:
% Support for Japanese
\setscript[hanzi]
\starttypescript[serif][sourcehanserifregular]
\definefontsynonym[Serif][name:sourcehanserifregular][features={aalt,hist,kern,liga}]
\stoptypescript
\definetypeface[sourcehanserifregular][rm][serif][sourcehanserifregular]
\define[1]\JA{\language[ja]{{\setscript[hanzi]\sourcehanserifregular #1}}}
You can try to play with this font setup, all settings within the setups-block are applied when you change the language. % font setup for english ... \definefontfamily [source] [rm] [Source Serif] \switchtobodyfont [source] % font setup for japanese \definefontfamily [source-han] [rm] [Source Han Serif] \startsetups [japanese] \switchtobodyfont [source-han] \setscript [hanzi] \stopsetups \setuplanguage [ja] [setups=japanese] \starttext English {\ja 日本語} \stoptext Wolfgang
participants (5)
-
Alan Braslau
-
Hans Hagen
-
luigi scarso
-
Richard Mahoney
-
Wolfgang Schuster