I use following in Latex to generate old factorial symbol:

\newcommand{\oldfact}[1]{%
\tikz[baseline]{\node[anchor=base,inner sep=0.3ex](mynode){\ensuremath{#1}};\draw(mynode.north west)--(mynode.south west)--(mynode.south east);\path[use as bounding box]($(mynode.south west)+(-0.3ex,-0.3ex)$)rectangle($(mynode.north east)+(0.3ex,0.3ex)$);}
}

But I do not know how to make it work in ConTeXt as I am not very familiar with ConTeXt.

If you see https://www.ntg.nl/maps/36/09.pdf page no. 28 then proof is defined as

\defineenumeration
[proof]
[ text=Proof,
number=no,
headstyle=italic,
title=no, %this is the default
closesymbol={\mathematics{\square}},
style=normal]

But the closessymbol does not work.

I will create a minimum working example and post it as soon as possible.