Re: [NTG-context] Files on a server

16 May 2018

      Beste Taco,

Ik probeer jouw oplossing maar er gebeurt nog iets raars. Ik krijg inderdaad 200 resp. 404 terug bij bestaande niet-bestaande Uri’s. Maar ik krijg wel een error die aan het eind van de run gegenereerd wordt. Heb jij enig idee waar ik dat zou moeten zoeken? Als ik de call naar \verbose uitschakel komt er geen foutmelding. Blijft er na de socket call iets achter wat nog opgeruimd had moeten worden?

Hier een deel van de context-log en de code fragmenten in kwestie. <takepersons> is de root node van het bewerkte bestand.

HVDM-PEV-TEST   > return code: 200
HVDM-PEV        > number of events processed: 1
HVDM-PEV        > ============= end of takepersons =============
backend         > xmp > using file '/Users/hansm/TeX/context-45/tex/texmf-context/tex/context/base/mkiv/lpdf-pdx.xml'
pages           > flushing realpage 1, userpage 1, subpage 1
close source    > level 2, order 13, name '/Users/hansm/Documents/Genealogie/familie-event.tex'
close source    > level 1, order 13, name '/Users/hansm/TeX/context-45/tex/texmf-context/tex/context/base/mkiv/cont-yes.mkiv'

lua error       > lua error on line 0 in file :

...-45/tex/texmf-context/tex/context/base/mkiv/core-uti.lua:92: attempt to index a number value (global 'status')

<empty file>

? 
Process aborted

Code in tex:
	\def\URIReturnCode#1{\ctxlua{tex.print(hvdm.urireturncode("#1"))}} 
	\verbose{HVDM-PEV-TEST}{return code: \URIReturnCode{\theurl}}
Code in lua:
	hvdm.urireturncode = function (theuri)
		content, status, authinfo = socket.http.request{
			method = "HEAD",
			url = theuri,
			}
		return tostring(status)
	end

Groet Hans
...
On 16 May 2018, at 14:23, Taco Hoekwater  wrote:
Or use luasocket, which is included in the luatex binary:
\startluacode
content, status, authinfo = socket.http.request{
   method = "HEAD",
   url = "http://hansvandermeer.myqnapcloud.com/archive/denhaag/hga-dtb-1869-6040.pdf” 
}
print (status)
\stopluacode
prints ‘404’ in this case.
Taco
...
On 16 May 2018, at 13:45, Henri Menke  wrote:
On 05/16/2018 11:23 PM, Hans van der Meer wrote:
...
I would be satisfied when a returned 404 error code will be handled
within a reasonable (configurable) time delay. As for redirection, there
I would not mind if it is not included.
Restrictions like that are not a problem for me, because this is for
building a pdf that accesses many internet located files. The check on
existence is therefore especially meant for spotting missing or misnamed
files.
You can use libcurl via ffi: https://curl.haxx.se/libcurl/
\startluacode
local ffi = require("ffi")
ffi.cdef[[
typedef struct Curl_easy CURL;
typedef int CURLcode;
typedef int CURLoption;
typedef int CURLINFO;
typedef size_t(*callback)(void *buffer, size_t size, size_t nmemb, void
*userp);
CURLcode curl_global_init(long flags);
CURL *curl_easy_init(void);
CURLcode curl_easy_setopt(CURL *curl, CURLoption option, ...);
CURLcode curl_easy_perform(CURL *curl);
CURLcode curl_easy_getinfo(CURL *curl, CURLINFO info, ...);
void curl_easy_cleanup(CURL *curl);
void curl_global_cleanup(void);
]]
-- magic numbers
local CURL_GLOBAL_DEFAULT = 3
local CURLOPT_URL = 10002
local CURLOPT_NOBODY = 44
local CURLOPT_HEADER = 42
local CURLOPT_WRITEFUNCTION = 20011
local CURLINFO_RESPONSE_CODE = 0x200000 + 2
local lcurl = ffi.load("curl")
local function discard(buffer, size, nmemb, userp)
  return size * nmemb
end
function check_status(url)
  lcurl.curl_global_init(CURL_GLOBAL_DEFAULT)
local curl = lcurl.curl_easy_init()
lcurl.curl_easy_setopt(curl, CURLOPT_URL, url)
  lcurl.curl_easy_setopt(curl, CURLOPT_NOBODY, ffi.cast("long",1))
  lcurl.curl_easy_setopt(curl, CURLOPT_HEADER, ffi.cast("long",1))
  lcurl.curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION,
ffi.cast("callback",discard));
local res = lcurl.curl_easy_perform(curl)
  local c_http_code = ffi.new("long[1]")
  lcurl.curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, c_http_code);
lcurl.curl_easy_cleanup(curl)
  lcurl.curl_global_cleanup()
return tonumber(c_http_code[0])
end
\stopluacode
\starttext
\ctxlua{context(check_status("http://example.com/"))}
\ctxlua{context(check_status("https://example.com/"))}
\ctxlua{context(check_status("https://example.com/xyz"))}
\stoptext
...
Hans van der Meer
...
On 16 May 2018, at 12:41, Henri Menke mailto:henrimenke@gmail.com> wrote:
On 05/16/2018 09:31 PM, Hans van der Meer wrote:
...
I tried to determine the existence of a file on the internet. See the
following macro call:
% Test if file exists.
       \edef\theurl{\linkprotocol://\urlbase\xmlatt{#1}{link}\thesuffix}
       \doiffileelse
       {\theurl}
       {\verbose{HVDM-PEV-TEST}{file exists: \theurl}}
       {\verbose{HVDM-PEV-TEST}{file does not exist: \theurl}}
It seems that \doiffileelse always results in true except for local
files where nonexistence is correctly handled.
For example the following url definitely does not exist but nevertheless
\doiffileelse reports its existence:
HVDM-PEV-TEST   > file exists:
http://hansvandermeer.myqnapcloud.com/archive/denhaag/hga-dtb-1869-6040.pdf
Is it possible that the \doiffile macros are extended for files on the
internet or a correponding set \doifurl is developed?
I'm not speaking for the devs here, but I don't think that this is going
to be included.  It would require that all webpages send proper 404
codes and timeout after reasonable delay.  Also how would you handle
redirects?  Should the ConTeXt run be stalled because checking for a
files existence on the web is caught in a redirect loop?
...
Hans van der Meer
___________________________________________________________________________________
If your question is of interest to others as well, please add an
entry to the Wiki!
maillist : ntg-context@ntg.nl mailto:ntg-context@ntg.nl /
http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry
to the Wiki!
maillist : ntg-context@ntg.nl
mailto:ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl
http://www.pragma-ade.nl/ / http://context.aanhet.net
http://context.aanhet.net/
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net http://contextgarden.net/
___________________________________________________________________________________
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________
Taco Hoekwater
Elvenkind BV
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________