Trimming strings and Lua string.trim question
I am trying to strip all leading and trailing whitespace from a string. I have tried two methods, neither of which is satisfactory. The first method uses \ignorespaces and \removeunwantedspaces. This fails to strip leading and trailing newlines. The second method uses the Lua string.strip function. The wiki description: Yields string with leading and trailing whitespace (spaces, horizontal and vertical tabs, newlines) removed suggests that this should do what I want, but I find that it removes internal whitespace as well as the leading and trailing whitespace. Can somebody suggest a better way of doing this? The following code demonstrates the problem: % macros=mkvi \starttexdefinition StringStrip #STRING \startluacode context(string.strip([==[#STRING]==])) \stopluacode \stoptexdefinition \long\def\test{ This is a test. \quad And it has an unexpected result. With \tex{ignorespaces} and \tex{removeunwantedspaces}, the newlines remain. With Lua {\tt string.strip}, the \tex{quad} at the end of a paragraph is preserved, but the other internal whitespace (including newlines) is gone! } \starttext \subject{ignorespaces and removeunwantedspaces} ¦\ignorespaces\test\removeunwantedspaces¦ \subject{Lua string.strip} ¦\StringStrip{\test}¦ \stoptext -- Rik
On 3/5/2014 5:45 AM, Rik Kabel wrote:
I am trying to strip all leading and trailing whitespace from a string. I have tried two methods, neither of which is satisfactory.
The first method uses \ignorespaces and \removeunwantedspaces. This fails to strip leading and trailing newlines.
The second method uses the Lua string.strip function. The wiki description:
Yields string with leading and trailing whitespace (spaces, horizontal and vertical tabs, newlines) removed
suggests that this should do what I want, but I find that it removes internal whitespace as well as the leading and trailing whitespace.
Can somebody suggest a better way of doing this?
The following code demonstrates the problem:
% macros=mkvi \starttexdefinition StringStrip #STRING \startluacode context(string.strip([==[#STRING]==])) \stopluacode \stoptexdefinition \long\def\test{
This is a test. \quad
And it has an unexpected result.
With \tex{ignorespaces} and \tex{removeunwantedspaces}, the newlines remain.
With Lua {\tt string.strip}, the \tex{quad} at the end of a paragraph is preserved, but the other internal whitespace (including newlines) is gone!
} \starttext \subject{ignorespaces and removeunwantedspaces} ¦\ignorespaces\test\removeunwantedspaces¦
\subject{Lua string.strip} ¦\StringStrip{\test}¦ \stoptext
striplong ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
On 2014-03-05 03:24, Hans Hagen wrote:
On 3/5/2014 5:45 AM, Rik Kabel wrote:
I am trying to strip all leading and trailing whitespace from a string. I have tried two methods, neither of which is satisfactory.
The first method uses \ignorespaces and \removeunwantedspaces. This fails to strip leading and trailing newlines.
The second method uses the Lua string.strip function. The wiki description:
Yields string with leading and trailing whitespace (spaces, horizontal and vertical tabs, newlines) removed
suggests that this should do what I want, but I find that it removes internal whitespace as well as the leading and trailing whitespace.
Can somebody suggest a better way of doing this?
The following code demonstrates the problem:
% macros=mkvi \starttexdefinition StringStrip #STRING \startluacode context(string.strip([==[#STRING]==])) \stopluacode \stoptexdefinition \long\def\test{
This is a test. \quad
And it has an unexpected result.
With \tex{ignorespaces} and \tex{removeunwantedspaces}, the newlines remain.
With Lua {\tt string.strip}, the \tex{quad} at the end of a paragraph is preserved, but the other internal whitespace (including newlines) is gone!
} \starttext \subject{ignorespaces and removeunwantedspaces} ¦\ignorespaces\test\removeunwantedspaces¦
\subject{Lua string.strip} ¦\StringStrip{\test}¦ \stoptext
striplong
It would be nice if striplong worked, but the example fails when string.striplong is used in place of string.strip. The log says: ! LuaTeX error [string "\directlua "]:1: attempt to call field 'striplong' (a nil value) stack traceback: [string "\directlua "]:1: in main chunk. Is the format of the call different than shown in the wiki, or is a different mechanism required to quote a potentially long string? -- Rik
Am 05.03.2014 um 16:07 schrieb Rik Kabel
striplong
It would be nice if striplong worked, but the example fails when string.striplong is used in place of string.strip. The log says:
! LuaTeX error [string "\directlua "]:1: attempt to call field 'striplong' (a nil value) stack traceback: [string "\directlua "]:1: in main chunk. Is the format of the call different than shown in the wiki, or is a different mechanism required to quote a potentially long string?
The name of the function is “strings.striplong”. Wolfgang
Am 05.03.2014 um 16:18 schrieb Wolfgang Schuster
Am 05.03.2014 um 16:07 schrieb Rik Kabel
: striplong
It would be nice if striplong worked, but the example fails when string.striplong is used in place of string.strip. The log says:
! LuaTeX error [string "\directlua "]:1: attempt to call field 'striplong' (a nil value) stack traceback: [string "\directlua "]:1: in main chunk. Is the format of the call different than shown in the wiki, or is a different mechanism required to quote a potentially long string?
The name of the function is “strings.striplong”.
This should be “utilities.strings.striplong” but it has no effect because when take a look at the saved string in Lua you can see that blank lines are removed from the text. Wolfgang
On 2014-03-05 10:54, Wolfgang Schuster wrote:
Am 05.03.2014 um 16:18 schrieb Wolfgang Schuster
mailto:schuster.wolfgang@gmail.com>: Am 05.03.2014 um 16:07 schrieb Rik Kabel
mailto:context@rik.users.panix.com>: striplong
It would be nice if striplong worked, but the example fails when string.striplong is used in place of string.strip. The log says:
! LuaTeX error [string "\directlua "]:1: attempt to call field 'striplong' (a nil value) stack traceback: [string "\directlua "]:1: in main chunk.
Is the format of the call different than shown in the wiki, or is a different mechanism required to quote a potentially long string?
The name of the function is "strings.striplong".
This should be "utilities.strings.striplong" but it has no effect because when take a look at the saved string in Lua you can see that blank lines are removed from the text.
Wolfgang
Indeed. This does build without error, but the utilities.strings.striplong result is worse than the string.strip result, with some additional whitespace at the end of the test string from the earlier example. So, is this a confirmed bug? -- Rik
On 3/6/2014 12:00 AM, Rik Kabel wrote:
On 2014-03-05 10:54, Wolfgang Schuster wrote:
Am 05.03.2014 um 16:18 schrieb Wolfgang Schuster
mailto:schuster.wolfgang@gmail.com>: Am 05.03.2014 um 16:07 schrieb Rik Kabel
mailto:context@rik.users.panix.com>: striplong
It would be nice if striplong worked, but the example fails when string.striplong is used in place of string.strip. The log says:
! LuaTeX error [string "\directlua "]:1: attempt to call field 'striplong' (a nil value) stack traceback: [string "\directlua "]:1: in main chunk.
Is the format of the call different than shown in the wiki, or is a different mechanism required to quote a potentially long string?
The name of the function is “strings.striplong”.
This should be “utilities.strings.striplong” but it has no effect because when take a look at the saved string in Lua you can see that blank lines are removed from the text.
Wolfgang
Indeed. This does build without error, but the utilities.strings.striplong result is worse than the string.strip result, with some additional whitespace at the end of the test string from the earlier example.
the definition of 'worse' depends on expectations and given the fact that in tex a newline or two newlines in a row have a different meaning it is just a guess what you want to achieve anyway, the next beta will have local str = table.concat( { " ", " aap", " noot mies", " ", " ", " zus wim jet", "zus wim jet", " zus wim jet", " ", }, "\n") utilities.strings.striplines(str,'prune'): [[aap noot mies zus wim jet zus wim jet zus wim jet]] utilities.strings.striplines(str,'prune and collapse'): [[aap noot mies zus wim jet zus wim jet zus wim jet]] utilities.strings.striplines(str,'prune and no empty'): [[aap noot mies zus wim jet zus wim jet zus wim jet]] utilities.strings.striplines(str,'retain'): [[ aap noot mies zus wim jet zus wim jet zus wim jet ]] utilities.strings.striplines(str,'retain and collapse'): [[ aap noot mies zus wim jet zus wim jet zus wim jet ]] utilities.strings.striplines(str,'retain and no empty'): [[ aap noot mies zus wim jet zus wim jet zus wim jet ]] so you can choose what you like Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
On 2014-03-06 05:38, Hans Hagen wrote:
On 3/6/2014 12:00 AM, Rik Kabel wrote:
On 2014-03-05 10:54, Wolfgang Schuster wrote:
Am 05.03.2014 um 16:18 schrieb Wolfgang Schuster
mailto:schuster.wolfgang@gmail.com>: Am 05.03.2014 um 16:07 schrieb Rik Kabel
mailto:context@rik.users.panix.com>: striplong
It would be nice if striplong worked, but the example fails when string.striplong is used in place of string.strip. The log says:
! LuaTeX error [string "\directlua "]:1: attempt to call field 'striplong' (a nil value) stack traceback: [string "\directlua "]:1: in main chunk.
Is the format of the call different than shown in the wiki, or is a different mechanism required to quote a potentially long string?
The name of the function is “strings.striplong”.
This should be “utilities.strings.striplong” but it has no effect because when take a look at the saved string in Lua you can see that blank lines are removed from the text.
Wolfgang
Indeed. This does build without error, but the utilities.strings.striplong result is worse than the string.strip result, with some additional whitespace at the end of the test string from the earlier example.
the definition of 'worse' depends on expectations and given the fact that in tex a newline or two newlines in a row have a different meaning it is just a guess what you want to achieve
anyway, the next beta will have
local str = table.concat( { " ", " aap", " noot mies", " ", " ", " zus wim jet", "zus wim jet", " zus wim jet", " ", }, "\n")
utilities.strings.striplines(str,'prune'):
[[aap noot mies
zus wim jet zus wim jet zus wim jet]]
utilities.strings.striplines(str,'prune and collapse'):
[[aap noot mies
zus wim jet zus wim jet zus wim jet]]
utilities.strings.striplines(str,'prune and no empty'):
[[aap noot mies zus wim jet zus wim jet zus wim jet]]
utilities.strings.striplines(str,'retain'):
[[ aap noot mies
zus wim jet zus wim jet zus wim jet ]]
utilities.strings.striplines(str,'retain and collapse'):
[[ aap noot mies
zus wim jet zus wim jet zus wim jet ]]
utilities.strings.striplines(str,'retain and no empty'):
[[ aap noot mies zus wim jet zus wim jet zus wim jet ]]
so you can choose what you like
Hans
indeed, beauty is in the eye of the beholder. Thank you for the flexible solution that allows each of us to find beauty. Could you please ensure that this handles text in buffers the same way that it handles other strings? I noticed that this is handled a bit differently, as the following shows,but perhaps it is my clumsy coding. The buffer result is closer to what I expect in terms of internal whitespace, but still does not show expected trimming of leading and trailing whitespace. That trimming, of course, is the point of the exercise. \def\StringsStripLong#1% {\ctxlua{context(utilities.strings.striplong([==[#1]==]))}} \def\StringStrip#1% {\ctxlua{context(string.strip([==[#1]==]))}} \startbuffer[testbuffer] B This is a test. And it has an unexpected result. \stopbuffer \long\def\testmacro{ M This is a test. And it has an unexpected result. } \starttext \subject{Lua string.strip passed TeX buffer} ¦\StringStrip{\getbuffer[testbuffer]}¦ \subject{Lua string.strip passed TeX macro} ¦\StringStrip{\testmacro}¦ \subject{Lua utilities.strings.striplong passed TeX buffer} ¦\StringsStripLong{\getbuffer[testbuffer]}¦ \subject{Lua utilities.strings.striplong passed TeX macro} ¦\StringsStripLong{\testmacro}¦ \stoptext -- Rik
Am 06.03.2014 um 16:04 schrieb Rik Kabel
Could you please ensure that this handles text in buffers the same way that it handles other strings? I noticed that this is handled a bit differently, as the following shows,but perhaps it is my clumsy coding. The buffer result is closer to what I expect in terms of internal whitespace, but still does not show expected trimming of leading and trailing whitespace. That trimming, of course, is the point of the exercise.
This can’t work because \getbuffer is a unexpandable command and what you pass to Lua is the string \getbuffer[testbuffer] and not the content of the buffer. When you want to manipulate the content of a buffer you have to access the content at the Lua side with the buffers.getcontent(…) function. BTW: You don’t need \long in MkIV because definitions use it by default. Wolfgang
On 2014-03-05 10:18, Wolfgang Schuster wrote:
Am 05.03.2014 um 16:07 schrieb Rik Kabel
mailto:context@rik.users.panix.com>: striplong
It would be nice if striplong worked, but the example fails when string.striplong is used in place of string.strip. The log says:
! LuaTeX error [string "\directlua "]:1: attempt to call field 'striplong' (a nil value) stack traceback: [string "\directlua "]:1: in main chunk.
Is the format of the call different than shown in the wiki, or is a different mechanism required to quote a potentially long string?
The name of the function is "strings.striplong".
Wolfgang
Thank you, Wolfgang. I have updated the wiki entry for this function. I have not been able to verify that other functions are similarly misidentified in the wiki, and have therefore not updated any others. Those who are more familiar with the functions may wish to review it. -- Rik
participants (3)
-
Hans Hagen
-
Rik Kabel
-
Wolfgang Schuster