Hi Roborob
you can boot RPi only from SD card. It contains not only the operating system components but also firware for the RPi's SoC, system configuration, etc.
Tiny Core Linux (TC) is not installed as ordinary Linux systems. It mounts extensions (packages with .tcz file name) which are stored in a /tce directory. In most cases this directory resides on the second partition of the SD card formatted as Linux ext/ext4 partion and big enough to accomodate extensions. This directory is also used to keep backups. But it is just a convention.
During boot TC is checking all available partitions (FAT/ext2/ext3/ext4) for /tce and will use the first found. It means, you can have an USB stick with /tce on ext4 partition. If no /tce on the SD card, system will use the USB stick. Same with HDD's or any other connected storage.
Notes:
External device must be ready during boot, otherwise TC will not find /tce. Use waitusb command line option. Start with waitusb=5 wich means 5 sec waiting for the USB. Increase it if necessary. Alternatively you can use UUID. Search Forum for details, it is not piCore specific.
HDD can consume high current during spinning up even if they are consuming much less during operation. HDD may require special USB cable, othervise you will loose voltage due to high current which may cause issues for the HDD. Considere that RPi's USB implementation is lets say poor, so I would not recommend to use an USB HDD. But it may work.
Just a private note, I do not see much reason to use USB stick or to play with extra power supplies, hubs, cables and loose an USB connector when 4G/8GB SD cards are cheap now and for the same price in few month you will get 16GB.