That's a great question about not using a full Xorg, and is basically a matter of preference - my major attraction to TC is to make the most out of minimalism for what I do. I learn so much from it, yet I'm not trying to be small for the sake of it.
What I found is that once I start going the full Xorg route, I basically end up duplicating a "distro" just bolting on add-ons until I'm bloated and tend not to learn anything other than distro building.
Although I *do* like a little bit of splash, say fltk-full and TTF fonts, and applications like Dillo or RXVT compiled to take advantage of them.
Not that there's anything wrong with duplicating a distro, TC is beautiful because the op can take it anywhere they like and of course the whole filesystem ethos of TC is cool. Goldilocks and just the right porridge for me.
A recent example is that for decades I used BC as my quickie desk calculator that comes with nearly every distro. But that is not in the busybox applet set but DC is!
Sooo... instead of installing BC, I learned DC (rpn cli calc) a bit and am totally happy with it. Whaddy'a know - an "app" from 1969/70 just prior to Unix (Thanks Bob Morris and Lorinda Cherry!) rocks.
Sorry for the long rant - it's just that when I go Xorg, the temptation to add yet another application is never ending.