Not sure what you mean by DSL, but your screenshot says Tiny Core so you're in the right place
I recall booting to Xvesa GUI with under 40MB RAM at one time, but my old hardware only gave me 800x600 resolution. There are many things to try. Reboot and check BIOS settings regarding graphics and see if you are able to manually allocate more memory to graphics.
If neither this or Xfbdev help then remove the Xfbdev entry from /etc/sysconfig/tcedir/onboot.lst, so you just have Xvesa. Reboot system and utilize 'text' boot code to reboot into CLI only. In CLI run the 'xsetup' command and try a different resolution. Then type 'startx' to test. If no good, i believe Ctrl-Alt-Backspace or Delete get you back to CLI to test another resolution.
My old hardware never had success with Xfbdev either, but my impression is that to use it Xvesa should be removed from onboot.lst and replaced with Xfbdev, then use the appropriate vga=XXX bootcode.
If you can't get a decent resolution/colour combination then you may need to go with Xorg, which might be a stretch given the limited hardware, but what the heck may as well try. With the same old hardware, Xorg allows me to run pretty much any resolution i want. You may or may not require a customize xorg.conf file. You didn't specity graphic hardware, make sure you pick the correct video driver if going with Xorg.
A reboot is necessary when switching between Xvesa, Xfbdev and Xorg.
Chapter 28 of the Into the Core book, regarding old hardware:
28.3. VESA support
Some older graphics cards don’t have proper support for the VESA
standard. This means that the standard Xvesa server might display at
a wrong resolution, with wrong colors, or fail to start altogether.
In these cases, the options are the framebuffer, and Xorg. To use a
framebuffer resolution, you need to add the vga=791 bootcode to
your bootloader’s config file (where 791 is a number specifying the
resolution and color depth - this particular one is 1024x768 at 16bit
color depth), and to install the Xfbdev server instead of Xvesa.
Table of common VESA resolutions:
640x480 800x600 1024x768 1280x1024
256 colors 769 771 773 775
16-bit 785 788 791 794
24-bit 786 789 792 795
Should the framebuffer also fail, or if non-VESA resolutions are
needed, you’ll need to install Xorg with a suitable driver.
There doesn’t exist a Xorg driver for all cards - check
online before trying.