There are so many different issues discussed, one could comment on any one of them.
Tiny Core should run on FAT32, just that errors are more likely on FAT32 than Ext2.
If you just use this usb drive in your own computer, you can install a program in Windows to be able to access Ext2. If you use in in many different computers, you may not be able to install it on all of them, so you may want to stick with FAT32.
The easiest way to install Tiny Core on a usb drive, is to run Tiny Core from the CD, connect to the internet, install dosfstools and syslinux, then run USB Install from the Control Panel. It will be set up automatically, and everything will work. You can select either FAT32 or Ext2. Then run from the usb drive, connect to the internet, and install programs you want on the usb drive.
If experienced users want to get more technical, they can create a FAT32 partition and an Ext2 partition. The FAT32 must be the first partition for Windows to access it. Install Tiny Core on the Ext2 partition. You can then access the FAT32 partition from both Tiny Core and Windows, and share files. This needs to be done manually, and cannot be done using USB Install.
If you already have Tiny Core installed, it is probably just a matter of minor modification, as mentioned by others. Try this first. If you can't get it working, try installing using USB Install.