Re: [NTG-context] Ruby 1.9.1 and non-ascii char parsing in .tui file

Hi all, I think I solved the problem. At least for my actual errors... I read the following net article about string coding in ruby 1.9 and up: http://blog.grayproductions.net/articles/ruby_19s_string With that info at hand, I made two >> brute-force "trial" patches << (read the above article to see why I call them "brute force" :-) ) in two of the ruby context files where problems were arising (original line numbers are shown): ### .../scripts/context/ruby/base/tex.rb ## (946) case str.chomp ===>>> str = str.force_encoding("ISO-8859-1") if RUBY_VERSION >= "1.9" case str.chomp ### .../scripts/context/ruby/base/texutil.rb ### (1033) case line.chomp ===>>> line = line.force_encoding("ISO-8859-1") if RUBY_VERSION >= "1.9" case line.chomp The error is due to the fact that, by default, ruby 1.9 considers strings as US-ASCII and complains when finding chars not in that encoding. I don't know how to solve the problem for people writing in other encoding which is not ISO-8859-1. I tried the above with UTF-8 instead of ISO-8859-1 and it didn't work. Finally I don't know if there are any other places (at least case expressions) in the context ruby scripts where the problem might also show up. Kind regards, J. Augusto On Mon, Aug 10, 2009 at 4:15 AM, Jose Augusto wrote:

Hi all,

Ok, here it goes. Atached are the files used in the test.

The problem as reported in the >> previous email << used the file with the offending chars wrapped in a main file, which was just:

\starttext \input zzz.tex \stoptext

That is, the offending chars were in zzz.tex.

In that example I noticed the error because the cross-refs in the equation numbering were not working. The parsing of the .tui file by ruby 1.9.1 failed. Then I saw the errors.

But then I made >> a single context file <<, which goes attached with the correct results (tui, tuo, pdf), obtained with ruby 1.8.7.

Howver, when i run ruby 1.9.1 with patch 129 (the last one), in this single tex file (attached) now the first pass don't work! Here is the result (in windows xp), which proves ruby 1.9.1 doesn't like the non US-ASCII chars :-)

F:\ANOS\ano09-10-pen\NotasProcSinal>texexec test1.tex TeXExec | processing document 'test1.tex' TeXExec | no ctx file found D:/Context/tex/texmf-context/SCRIPTS/CONTEXT/ruby/base/tex.rb:946:in `===': invalid byte sequence in US-ASCII (ArgumentError) from D:/Context/tex/texmf-context/SCRIPTS/CONTEXT/ruby/base/tex.rb:946: in `scantexcontent' from D:/Context/tex/texmf-context/SCRIPTS/CONTEXT/ruby/base/tex.rb:1907: in `processfile' from D:/Context/tex/texmf-context/SCRIPTS/CONTEXT/ruby/base/tex.rb:1143: in `block (2 levels) in processtex' from D:/Context/tex/texmf-context/SCRIPTS/CONTEXT/ruby/base/tex.rb:1133: in `timedrun' from D:/Context/tex/texmf-context/SCRIPTS/CONTEXT/ruby/base/tex.rb:1142: in `block in processtex' from D:/Context/tex/texmf-context/SCRIPTS/CONTEXT/ruby/base/tex.rb:1139: in `each' from D:/Context/tex/texmf-context/SCRIPTS/CONTEXT/ruby/base/tex.rb:1139: in `processtex' from D:/Context/tex/texmf-context/scripts/context/ruby/texexec.rb:63:in`process' from D:/Context/tex/texmf-context/scripts/context/ruby/texexec.rb:53:in `main' from D:/Context/tex/texmf-context/SCRIPTS/CONTEXT/ruby/base/switch.rb:133:in `execute' from D:/Context/tex/texmf-context/scripts/context/ruby/texexec.rb:787:in `<main>'

Thanks all for your interest,

Kind regards

J. Augusto

On Sun, Aug 9, 2009 at 8:57 PM, Hans Hagen wrote:

...

Jose Augusto wrote:

when /^c (.*)$/o then @plugins.reader('MyCommands', [$1])

...

what if you remove the o (/o)

can you find out what changed between 1.8 and 1.9 ... actually 1.9 is the stepping stone to 2.0 and 2 versions can be incompatible to 1 versions

also, can you make a test file so that we can see if there's a platform dependency?

----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net

___________________________________________________________________________________