# [NTG-context] Count (and limit) glyphs per line?

Benjamin Buchmuller benjamin.buchmuller at gmail.com
Sun Jun 26 17:59:01 CEST 2022

Hi Max,

Thank you so much for your help and pointing me to the documents; always a lot of things to learn in TeX!

I'm afraid that including the hyphen width doesn't solve the issue yet. It seems to move the problem to other parts of the text. My guess is that one could equivalently have said "local max_length = 111", right?

I made the following MWE (reproducible also online) to illustrate what I see:

* Here, instead of a breaking point, the trouble is caused by not being able to break it. This causes the next line to be underfull. (I get a lot of these, but also some with hyphenated breakpoints, in my own document. Maybe the insertion point of the penalty/breaking bonus needs to move up?)

* Running with hsize only makes the problem worse in itemizations, so I think localhsize is the way to go. My guess, localhsize is the width of the "text" part of a paragraph, for example, excluding the symbols in the itemization.

(More thoughts below)

\startluacode
local max_length = 112

local glyph_id = node.id "glyph"
local disc_id = node.id "disc"
local glue_id = node.id "glue"

local hyphen = node.new "glyph"
hyphen.char = language.prehyphenchar(0)
hyphen.font = font.current()
local width = hyphen.width
node.free(hyphen)

local chars = 0
while n do
if n.id == glyph_id or n.id == glue_id then
chars = chars + 1
width = width + n.width - (n.shrink or 0)
end

local localhsize = tex.dimen["textwidth"]

if tex.dimen["localhsize"] > 0 then
localhsize = tex.dimen["localhsize"]
end

if chars >= max_length or width > localhsize then
local back_chars = 0
local end_disc = nil

while n do
if n.id == glue_id then
local penalty = node.new "penalty"
penalty.penalty = -10000
break
end

if not end_disc and n.id == disc_id then
end_disc = n
end

if end_disc and back_chars >= 5 then
end_disc.penalty = -10000
break
end

if n.id == glyph_id then
back_chars = back_chars + 1
end

n = n.prev
end

width = 0
chars = 0
end

n = n.next
end

end

"processors",
"before",
"userdata.limiter"
)
\stopluacode

\setuppapersize[A5]
\showframe

\starttext

This is text width:
\ctxlua{context(tex.dimen["textwidth"])}

This is hsize:
\ctxlua{context(tex.dimen["hsize"])}

This is localhsize:
\ctxlua{context(tex.dimen["localhsize"])}

\startitemize[width=5em]

\item Thus, I came to the conclusion that the \hbox{de­signer} of a new system must not only be the implementer and first large--scale user; the de­ signer should also write the first user manual.

\item \samplefile{knuth}

This is text width:
\ctxlua{context(tex.dimen["textwidth"])}

This is hsize:
\ctxlua{context(tex.dimen["hsize"])}

This is localhsize:
\ctxlua{context(tex.dimen["localhsize"])}

\stopitemize

\stoptext

I'm wondering if I do understand the second while loop correctly:

* Once we find the node that exceeds either the character limit or the (local-)hsize (glyphs and glues summed, for disc we add hyphen.width, do we?), then we insert an incredibly good breaking point for a new line. And exit the loop.

* The other cases still seem a bit obscure to me, and I tried to trace where each of them might be triggered:

if n.id == glue_id then
local penalty = node.new "penalty"
penalty.penalty = -10000
context.inrightmargin("glue")
break
end

if not end_disc and n.id == disc_id then
context.inrightmargin("disc")
end_disc = n
end

if end_disc and back_chars >= 5 then
context.inrightmargin("end")
end_disc.penalty = -10000
break
end

if n.id == glyph_id then
context.inrightmargin("glyph")
back_chars = back_chars + 1
end

I'm maybe doing this wrong, but I see these conditions triggered more often than probably expected for a 25 line document?

local count_me = 0

...

if chars >= max_length or width > localhsize then
local back_chars = 0
local end_disc = nil

while n do
local check = "glyph"

count_me = count_me + 1
if n.id == glue_id then
local penalty = node.new "penalty"
penalty.penalty = -10000
context.inrightmargin("\\color[red]{" .. string.rep("_", count_me) .. count_me .. "}")
break
end

if not end_disc and n.id == disc_id then
end_disc = n
end

--
if end_disc and back_chars >= 5 then
context.inrightmargin("\\color[blue]{" .. string.rep("_", count_me) .. count_me .. "}")
end_disc.penalty = -10000
break
end

if n.id == glyph_id then
context.inrightmargin("\\color[black]{" .. string.rep("_", count_me) .. count_me .. "}")
back_chars = back_chars + 1
end

n = n.prev
end

Many thanks again!

Benjamin

> On Jun 25, 2022, at 17:40, Max Chernoff <mseven at telus.net> wrote:
>
>> It's also a very insightful example of how to use and inject Lua code in the TeX output routine.
>
> This is injecting Lua code before the paragraph builder, not in the output routine. Something like https://tex.stackexchange.com/a/644613/270600 or my module "lua-widow-control" would be an example of Lua code in the output routine.
>
>> Do you mind if I add it to the wiki? (Probably under "Wrapping".)
>
> Sure
>
>> However, tex.localhsize (or tex.dimen["localhsize"]) is 0 when the document is initialized. (Maybe a more sensible default would be textwidth rather than 0?)
>> 	local localhsize = tex.dimen["textwidth"]
>>
>> 	if tex.dimen["localhsize"] > 0 then
>> 		localhsize = tex.dimen["localhsize"]
>> 	end
>>         if chars >= max_length or width > localhsize then
>
> I don't think that's necessary. \hsize is a primitive TeX parameter that sets the width of the paragraph. It may be zero at the start of the document, but it is definitely non-zero by the end of every paragraph.
>
> The Lua function gets the current value of \hsize at the end of every paragraph, so it should be using the exact same value that TeX's paragraph builder uses, meaning that it should account for itemizations and such. I'm not really sure what \localhsize is, but it's probably similar to \hsize.
>> (2) I'm (now?) running into trouble with hyphenation.
>> In my own document, I also get lines with only a single character or hboxed group. I assume, this is because the hyphen is not counted and pushes the remainder to a new line where the intended breakpoint again starts another one.
>
> Try this:
>
>    \startluacode
>        local max_length = 112
>
>        local glyph_id = node.id "glyph"
>        local disc_id = node.id "disc"
>        local glue_id = node.id "glue"
>
>
>            local hyphen = node.new "glyph"
>            hyphen.char = language.prehyphenchar(0)
>            hyphen.font = font.current()
>            local width = hyphen.width
>            node.free(hyphen)
>
>            local chars = 0
>            while n do
>                if n.id == glyph_id or n.id == glue_id then
>                    chars = chars + 1
>                    width = width + n.width - (n.shrink or 0)
>                end
>
>                if chars >= max_length or width > tex.hsize then
>                    local back_chars = 0
>                    local end_disc = nil
>
>                    while n do
>                        if n.id == glue_id then
>                            local penalty = node.new "penalty"
>                            penalty.penalty = -10000
>                            break
>                        end
>
>                        if not end_disc and n.id == disc_id then
>                            end_disc = n
>                        end
>
>                        if end_disc and back_chars >= 5 then
>                            end_disc.penalty = -10000
>                            break
>                        end
>
>                        if n.id == glyph_id then
>                            back_chars = back_chars + 1
>                        end
>
>                        n = n.prev
>                    end
>
>                    width = 0
>                    chars = 0
>                end
>
>                n = n.next
>            end
>
>        end
>
>            "processors",
>            "before",
>            "userdata.limiter"
>        )
>    \stopluacode
>
> I've just added the width of a hyphen to the accumulated width. Let me know if this works; if not, there's a more complex fix that I can try.
>
>> Unfortunately, I don't know what to change; I know a bit about "glyph" and "glue", but what is "disc" and would it help here?
>
> "disc" nodes are "discretionaries", which are usually potential hyphens. See "The TeXbook" (page 95) or "TeX by Topic" (https://texdoc.org/serve/texbytopic/0#subsection.19.3.1) for details on the TeX side, or the LuaMetaTeX manual (https://www.pragma-ade.com/general/manuals/luametatex.pdf#%231205) for details on the Lua side.
>
> -- Max