Some updates ...
I found the sources for flwm at
https://github.com/tinycorelinux/flwm.git, but I couldn't find any relationship between the tcl versions and the commits in the repository. I found the current sources for fltk, but none for 5.x.
I generated flwm and flwm_topside from the sources and ran them. As before, my demo application does not work with flwm_topside but "works" with flwm. As before aterm with an utf-8 encoded title works with flwm_topside, however, I did't try before, it doesn't work with flwm.
So I added some fprintf's showing the to be printed title. For both, flwm_topside and flwm, it showed the ISO-LATIN1 character 'ü' for my demo application and the utf-8 encoded 'ü' for the aterm.
That is, flwm_topside prints utf-8 encoded titles and flwm doesn't. And vice versa, flwm prints ISO-LATIN1 characters and flwn_topside doesn't.
Obviously there are different ways how the title set in aterm and the title set in my gdk demo application makes it to the windows manager, flwm. I didn't look at aterm. Gtk sets the title in X11 atoms, XA_WM_NAME and _NET_WM_NAME. The latter is set to the utf-8 encoded title, the first to a "compound_text" type of text, which here is a ISO-LATIN1 encoded text.
Flwm only retrieves the XA_WM_NAME atom. This explains why I see the ISO-LATIN1 character 'ü' in my debug output. This also means that aterm sets the utf-8 title in XA_WM_NAME. (I assume, it sets whatever it finds in the -title option.)
Retrieving the _NET_WM_NAME atom is not possible, as it is not supported by the default X11 server, Xvesa. (Which to me also means, that gtk fails to set it, but that doesn't matter, here.)
Because my demo application works on 5.x of tcl, it is very likely that flwm_topside for that version printed the ISO-LATIN1 title. As it
no longer does, it is either a regression in flwm_topside or in fltk's fl_draw(), which is used, here - at least in the sources I have. Without having the sources of flwm and fltk for 5.x it is hard to tell where the regression is.
In the current version, for the title, flwm (left side) uses draw_rotated90(), defined in Rotated.C. (fltk is configured with --disable-xft, so there is no HAVE_XFT defined when compiling flwm.)
So I changed the sources for flwm_topside to use similar code as in draw_rotated90() that is finally calling draw_rotated() with angle 0 for the title. Now flwm_topside behaves as flwm, which is an acceptable behavior for me (and aterm with an utf-8 title no longer
works, which is OK for me, too).
It seems the default X11 server and flwm do not really support utf-8.