You can easily set up e.g. an encrypted home dir, see the cryptsetup.tcz extension.
I don't recommend the ubuntu way. Instead manually set up multiple separate security levels protected by different passphrases, and only one encrypted partition should be mounted at the same time.
You could have tinycore on a flash drive and make it mount the crypted drives on boot asking you for a passphrase to access the disks that would otherwise be useless to other people.
The stick would also be useless cause it only has tinycorelinux and official packages. You'd put your private data on the crypted home dir that is not available without the other disk + passphrase.