I prefer to use hard drives.
USB drives have a limited life, maybe several years.
It is not the cost of the USB drive that is of concern. It is the fact that when it fails, you lose all of your personal files.
I know others have different opinions.
I have actually been running Tiny Core, and previously other Linux distros, on computers which did not cost me anything for several years. Computers which other people discarded because they are a little slow to run Windows XP. Thousands are being thrown away every year. These computers have hard drives. Some have been working, others have needed to be repaired.
For example, one of the better free computers is 800 mHz with a 40 gb hard drive. This one had a power supply not working. I replaced this with a power supply from another free computer.
It came with 256 mb of ram. I added another 512 from another computer in which the motherboard failed.
Thousands of computers like this are being thrown out every year. It may be difficult for most people to get hold of them.
Some people are selling used computers like this at excessively high prices to people who don't understand.