Hi, there are multiple issues memory issues in tprint: If the buffer passed to tprint only contains of to be escaped characters, the `xmalloc(strlen(sss)*3)` is too small: The final `\0` can not be written, so this causes a write into unallocated memory. If t_flush_buffer is called for such a buffer at the last position, there is an additional `\n` written, so `xmalloc(strlen(sss)*3+2)` is needed. Additionally, the test `if (*buffer)` is used to test if there is something to print left. If the parameter is empty, `*buffer` is never written, so valgrind complains about `if (*buffer)` depending on uninitialized memory. Instead, `if (i)` can be used: `i` is the length of the written part of the buffer, so `i==0` iff there is nothing to write. A related problem is with `max_print_line`: If the point where the line should be broken falls inside the escape sequence of an escaped character, the line limit is ignored. This can be fixed by adding if (term_offset+2>=max_print_line) { wterm_cr(); term_offset=0; } as in `wterm_char` also in tprint above `buffer[i++] = '^';`. A full patch fixing all three issues is attached. Best regards Marcel Krüger
On 3/28/2019 3:57 PM, Marcel Krüger wrote:
Hi,
there are multiple issues memory issues in tprint:
If the buffer passed to tprint only contains of to be escaped characters, the `xmalloc(strlen(sss)*3)` is too small: The final `\0` can not be written, so this causes a write into unallocated memory. If t_flush_buffer is called for such a buffer at the last position, there is an additional `\n` written, so `xmalloc(strlen(sss)*3+2)` is needed.
Additionally, the test `if (*buffer)` is used to test if there is something to print left. If the parameter is empty, `*buffer` is never written, so valgrind complains about `if (*buffer)` depending on uninitialized memory. Instead, `if (i)` can be used: `i` is the length of the written part of the buffer, so `i==0` iff there is nothing to write.
A related problem is with `max_print_line`: If the point where the line should be broken falls inside the escape sequence of an escaped character, the line limit is ignored. This can be fixed by adding
if (term_offset+2>=max_print_line) { wterm_cr(); term_offset=0; }
as in `wterm_char` also in tprint above `buffer[i++] = '^';`.
A full patch fixing all three issues is attached. Can you give a plain tex example where the fault happens? Normally the max line length kicks in (3 chars) before such an overflow.
(btw, nothing fundamental will change in the texlive code freeze frame) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
---- On Thu, 28 Mar 2019 18:53:41 +0100 Hans Hagen
On 3/28/2019 3:57 PM, Marcel Krüger wrote:
Hi,
there are multiple issues memory issues in tprint:
If the buffer passed to tprint only contains of to be escaped characters, the `xmalloc(strlen(sss)*3)` is too small: The final `\0` can not be written, so this causes a write into unallocated memory. If t_flush_buffer is called for such a buffer at the last position, there is an additional `\n` written, so `xmalloc(strlen(sss)*3+2)` is needed.
Additionally, the test `if (*buffer)` is used to test if there is something to print left. If the parameter is empty, `*buffer` is never written, so valgrind complains about `if (*buffer)` depending on uninitialized memory. Instead, `if (i)` can be used: `i` is the length of the written part of the buffer, so `i==0` iff there is nothing to write.
A related problem is with `max_print_line`: If the point where the line should be broken falls inside the escape sequence of an escaped character, the line limit is ignored. This can be fixed by adding
if (term_offset+2>=max_print_line) { wterm_cr(); term_offset=0; }
as in `wterm_char` also in tprint above `buffer[i++] = '^';`.
A full patch fixing all three issues is attached. Can you give a plain tex example where the fault happens? Normally the max line length kicks in (3 chars) before such an overflow.
Try \let\3\relax \directlua{ texio.write'\3'% Only to be escaped characters -> valgrind complains about write into unallocated memory texio.write_nl'' for i=1,kpse.var_value'max_print_line'-3 do texio.write'.' end texio.write'\3' % Here LuaTex writes 2 and not just 1 byte after it's allocated memory area for i=1,kpse.var_value'max_print_line'-1 do texio.write'.' end texio.write'\3' % The line break would fall inside of the escape sequence -> Line limit ignored for i=1,2*kpse.var_value'max_print_line' do texio.write'.' end % This should normally not fit into one line } \bye
(btw, nothing fundamental will change in the texlive code freeze frame)
Right, it isn't urgent anyway, I just wanted to clean up my valgrind output... Best regards Marcel Krüger
Hans
----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl ----------------------------------------------------------------- _______________________________________________ dev-luatex mailing list dev-luatex@ntg.nl https://mailman.ntg.nl/mailman/listinfo/dev-luatex
On Thu, Mar 28, 2019 at 10:55 PM Marcel Krüger
On 3/28/2019 3:57 PM, Marcel Krüger wrote:
Hi,
there are multiple issues memory issues in tprint:
If the buffer passed to tprint only contains of to be escaped characters, the `xmalloc(strlen(sss)*3)` is too small: The final `\0` can not be written, so this causes a write into unallocated memory. If t_flush_buffer is called for such a buffer at the last position,
Additionally, the test `if (*buffer)` is used to test if there is
something to print left. If the parameter is empty, `*buffer` is never written, so valgrind complains about `if (*buffer)` depending on uninitialized memory. Instead, `if (i)` can be used: `i` is the length of
---- On Thu, 28 Mar 2019 18:53:41 +0100 Hans Hagen
wrote ---- there is an additional `\n` written, so `xmalloc(strlen(sss)*3+2)` is needed. the written part of the buffer, so `i==0` iff there is nothing to write. A related problem is with `max_print_line`: If the point where the line should be broken falls inside the escape
sequence of an escaped character,
the line limit is ignored. This can be fixed by adding
if (term_offset+2>=max_print_line) { wterm_cr(); term_offset=0; }
as in `wterm_char` also in tprint above `buffer[i++] = '^';`.
A full patch fixing all three issues is attached. Can you give a plain tex example where the fault happens? Normally the max line length kicks in (3 chars) before such an overflow.
Try
\let\3\relax \directlua{ texio.write'\3'% Only to be escaped characters -> valgrind complains about write into unallocated memory
texio.write_nl''
for i=1,kpse.var_value'max_print_line'-3 do texio.write'.' end texio.write'\3' % Here LuaTex writes 2 and not just 1 byte after it's allocated memory area
for i=1,kpse.var_value'max_print_line'-1 do texio.write'.' end texio.write'\3' % The line break would fall inside of the escape sequence -> Line limit ignored for i=1,2*kpse.var_value'max_print_line' do texio.write'.' end % This should normally not fit into one line } \bye
Ok, thank you very much for the example. -- luigi
On Thu, Mar 28, 2019 at 10:55 PM Marcel Krüger
\let\3\relax \directlua{ texio.write'\3'% Only to be escaped characters -> valgrind complains about write into unallocated memory
texio.write_nl''
for i=1,kpse.var_value'max_print_line'-3 do texio.write'.' end texio.write'\3' % Here LuaTex writes 2 and not just 1 byte after it's allocated memory area
for i=1,kpse.var_value'max_print_line'-1 do texio.write'.' end texio.write'\3' % The line break would fall inside of the escape sequence -> Line limit ignored for i=1,2*kpse.var_value'max_print_line' do texio.write'.' end % This should normally not fit into one line } \bye
How do you run valgrind ? Which version ? Do you compile luatex with standard -O2 ? -- luigi
---- On Fri, 29 Mar 2019 08:13:19 +0100 luigi scarso
On Thu, Mar 28, 2019 at 10:55 PM Marcel Krüger
wrote: \let\3\relax \directlua{ texio.write'\3'% Only to be escaped characters -> valgrind complains about write into unallocated memory
texio.write_nl''
for i=1,kpse.var_value'max_print_line'-3 do texio.write'.' end texio.write'\3' % Here LuaTex writes 2 and not just 1 byte after it's allocated memory area
for i=1,kpse.var_value'max_print_line'-1 do texio.write'.' end texio.write'\3' % The line break would fall inside of the escape sequence -> Line limit ignored for i=1,2*kpse.var_value'max_print_line' do texio.write'.' end % This should normally not fit into one line } \bye
How do you run valgrind ? Which version ? Do you compile luatex with standard -O2 ? -- luigi
My test were done with (Arch) Linux, Kernel 5.0.4-arch1-1-ARCH glibc 2.28-5 valgrind-3.14.0 The tests were done with a debug build (`./build.sh --debug`) a regular build (`./build.sh` w/o further options) and the current TeXLive pretest version.
On Fri, Mar 29, 2019 at 9:26 AM Marcel Krüger
My test were done with
(Arch) Linux, Kernel 5.0.4-arch1-1-ARCH glibc 2.28-5 valgrind-3.14.0
The tests were done with a debug build (`./build.sh --debug`) a regular build (`./build.sh` w/o further options) and the current TeXLive pretest version.
Ok; following valgrind [valgrind-options] [your-program] [your-program-options] what are valgrind-options you are using ? -- luigi
---- On Fri, 29 Mar 2019 09:30:39 +0100 luigi scarso
On Fri, Mar 29, 2019 at 9:26 AM Marcel Krüger
wrote: My test were done with
(Arch) Linux, Kernel 5.0.4-arch1-1-ARCH glibc 2.28-5 valgrind-3.14.0
The tests were done with a debug build (`./build.sh --debug`) a regular build (`./build.sh` w/o further options) and the current TeXLive pretest version.
Ok; following valgrind [valgrind-options] [your-program] [your-program-options] what are valgrind-options you are using ?
-- luigi
Only default options: valgrind luatex weird for the TeX file weird.tex. I attach the full valgrind output from my system.
participants (3)
-
Hans Hagen
-
luigi scarso
-
Marcel Krüger