some notes on the philosophy behind TC

[jamtat]

I generally like smaller distros aimed at running on older hardware and enjoy exploring alternatives in general, so I was attracted to TC. I should also point out that I was a regular follower of TC's predecessor (DSL) and installed and used several releases of that distro. Finally, I've been using Linux pretty much full time for about 10 years and have a fair amount of experience administering my own computers.

I experimented around a bit when I first started using Linux: first with Mandrake, then with Red Hat, then Redmond Linux (remember that one?), Knoppix, and some Slackware variants. I finally settled on Debian, just before Ubuntu hit the scene. I finally switched over to Ubuntu, though in recent years I've also been using Arch. So, though I don't have any formal training in computers, nor do I work in any computer-related job, I have a fair amount of experience with them and with Linux through pursuing this as a sort of hobby.

Despite my decent grounding in Linux I'm still struggling to understand the TC philosophy and approach. One aspect of it I'd like to take issue with in this post is what is referred to as "system rot."

It seems TC has been constructed as it has in large part to address the perceived problem of system rot. Right off the bat I have a problem with that aspect of the philosophy. Why? Because no Linux system I've set up and used has ever suffered from system rot. Lots of Windows systems I've set up and used have suffered from system rot, but no Linux system I've ever set up and used has. Let me explain.

I was a Windows user prior to switching to Linux. One of the things that bugged me about Windows was the way the system would get corrupted. It would only work really well right after a pristine installation. Once you started installing programs and using the system more, performance would degrade. Eventually the system would be functioning so poorly that it would be necessary to save all important data to some external storage and wipe out the Windows installation, replacing it with a new one. Then the system would begin functioning properly again--that is, until, after another extended period of program installations and use it would again become virtually unusable, necessitating once again a data back-up and re-installation of the operating system. All Windows systems I used (up to W2K, which is about the time I switched to Linux) functioned like this. When I think of the phrase "system rot," this is what I think of, i.e., a system that, through normal use has become so buggy that it's a pain to use it.

But since I've switched to Linux, this problem has gone away. I've never had any Linux system I've set up and used in the last 10 years degrade in performance so badly the a re-installation of the operating system was required. I've made some mistakes in system administration that necessitated re-installation of the operating system, but most of the re-installing I've done has been for experimentation purposes, not because the system had become unusable. On the contrary, I'm constantly amazed at how flexible and resilient the Linux systems I've set up and used are.

Take, for example, the system I'm now using. This is an Arch Linux installation that was originally installed a little ovcer 2 years ago. The original installation was on a completely different system--a Sempron 3000+. I had some hard drive problems with the original system (failing hardware) and had to replace the drive. By that time, the drive had been in the computer and in use for about a year. I bought a larger HD and copied over all the data from the failing drive, stuck the new drive in the machine, and was up and running. There were a few hiccups, to be sure. But with a bit of nursing I was able to restore the system to perfectly normal operation--pretty much like it operated right after I had installed Arch.

But later I started to experience another hardware failure. This time it was bad caps (system spontaneously powering down). Since the hardware was getting a bit dated, I decided to start looking into a more major upgrade. What I ended up doing was buying a new mobo/CPU combo (dual core) and power supply, and swapping that in. All I needed to do to get the system to boot and run normally was to switch kernels, which I did with the agency of an Arch boot CD. Voila, I now have an almost completely different hardware system, all with the original operating system I installed over two years ago. And it still functions just as well as it did after the initial installation. It also has been running pretty much 24 hours a day the whole time. Talk about resiliency!

So, where's the system rot here? I do update my system from the Arch repositories, though not on a regular schedule. I've done quite a few major upgrades (pacman -Syu, roughly equivalent to Debian/Ubuntu's apt-get dist-upgrade) that way since setting up the system. Now, I would not dispute the fact that I have myriad files strewn about the system that may not be crucial to its operation. In fact, it amazing to see the sheer number, the thousands of discrete files this system contains.

Could the system use some clean-up to prune out unnecessary/unneeded files? It probably couldn't hurt anything. But what would be the benefit? Given that I've seen no degradation in performance since the initial installation of this OS, I don't think cleaning up the system would be likely to help system performance. In fact, by all indications I could go on using this system and its successors just as I have for the past 2 years--swapping out major or minor pieces of hardware as needed--for the foreseeable future.

So, have I misunderstood what is "system rot?" Can it be something that, while taking place on someone's computer (say, mine), has no discernable effect beyond an appraent growth in the number fo files? Can some real-world examples (other than the Windows examples well known to me and that I've cited above) of it be given? Is addressing the problem of system rot really as essential to the TC philosophy and implementation as it has seemed to me to be?

Thanks,
James

[bmarkus]

I read your mail few times but still do not understand what is your issue and what are you explaining as TC specific.

[gerald_clark]

System rot I have seen is often caused by installing software from multiple sources.
Packages replacing utilities and libraries used by other packages can cause deterioration
in overall stability.  Simply removing an offending package can be impossible, or fatal.
By installing from scratch on every boot, the simple removal of an extension from onboot.lst
will bring the system back to the state it was before the package was added.

[jamtat]

I read your mail few times but still do not understand what is your issue and what are you explaining as TC specific.

Have you looked at this page: http://tinycorelinux.com/concepts.html ? Do you see the phrase "system rot" there? I can only come away from reading that page with the understanding that TC was designed, in large part, to address the perceived problem of system rot. What do you understand with regard to system rot as it's written about on that page?

As for me, I'm saying that system rot, as I understand it, has not been a problem on any Linux system I've ever set up and used. Just using Linux (instead of Windows) has taken care of the system rot problems I've had. But the TC philosophy apparently goes beyond this: it doesn't just say "use Linux and your system rot problems will be solved," but it seems to say "use this special brand of Linux, with this special design philosophy (e.g., installing applications to looped file systems), to resolve system rot."

That simplifies things a bit, doesn't it?

James

PS The TC philosophy reminds me a bit of the micro-kernel side of the debate that played out between Tannenbaum and Linus years ago. Which code should be allowed to run in the most privileged mode, and which should be relegated to less privileged (or to user space)? With Linux we get these gigantic, monolithic kernels, while Minix and Mach kernels are much smaller--and supposedly the systems are more stable, since there are fewer processes able to take the system down.

[jamtat]

System rot I have seen is often caused by installing software from multiple sources.

Do you mean, for example, installing Red Hat packages onto a Debian system? E.g., using alien? Or are you referring to compiling from source?

Packages replacing utilities and libraries used by other packages can cause deterioration
in overall stability.  Simply removing an offending package can be impossible, or fatal.

I've never had this (deterioration in stability) happen in my ten years of installing and using Linux. And I've tried many distros and have used many different systems. Has it happened to you? If so, what distro were you using?

James

[thane]

My understanding is that the philosophy behind TC is basically two-fold: a very small basic system that permits the user to install only those applications (extensions) that he/she wishes; and a careful separation of static data (e.g. the extensions) from dynamic data (e.g. user preference files, browsing history, which are saved in mydata) which are saved separately from the extension files. Because the extension directory/files are not updated during normal use these are presumably not degraded over time.

[jamtat]

My understanding is that the philosophy behind TC is basically two-fold: a very small basic system that permits the user to install only those applications (extensions) that he/she wishes;

But the smallness factor is related to the notion of system rot, isn't it? At least as it's explained at http://tinycorelinux.com/concepts.html , smallness and pristineness (non-rottedness) seem to be carefully linked.

and a careful separation of static data (e.g. the extensions) from dynamic data (e.g. user preference files, browsing history, which are saved in mydata) which are saved separately from the extension files. Because the extension directory/files are not updated during normal use these are presumably not degraded over time.

Ok. So there are mechanisms for reducing system rot even outside the core, right?

James

[curaga]

System rot I have seen is often caused by installing software from multiple sources.

Do you mean, for example, installing Red Hat packages onto a Debian system? E.g., using alien? Or are you referring to compiling from source?

It means anything outside official channels for the distro; from source, to other formats (deb -> rpm), to same format but for different distro (rpm for suse on fedora), to same format binary packages (skype).

[danielibarnes]

The wiki uses the phrase "inevitable system rot." As I understand it, James is saying that at least in his experience, system rot is not an issue.

Of course, this effect will differ with each individual's experience. For every person that has never seen system rot, there is someone whose system rots before their very eyes. The person's use of the system (installing, uninstalling) might not be a factor at all. It could be faulty hardware.

What Tiny Core provides that other installable operating systems do not, is the ability to make system rot impossible.

When you boot using only the kernel, root filesystem, and your extensions, you are as close as you can get to having the exact same setup everytime. Personally, that is why I find Tiny Core uniquely valuable. Consistency.

[jamtat]

The wiki uses the phrase "inevitable system rot." As I understand it, James is saying that at least in his experience, system rot is not an issue.

Yeah, what I'm saying is that I've only seen it on Windows systems. Never seen it on a Linux system--certainly not one I've installed and/or used over time.

Of course, this effect will differ with each individual's experience. For every person that has never seen system rot, there is someone whose system rots before their very eyes. The person's use of the system (installing, uninstalling) might not be a factor at all. It could be faulty hardware.

I've seen hardware issues. I had a Debian system that would spontaneously lock on me a few years ago, for example. But as soon as I stopped relying on the system's RAM for video and stuck in a real video card with its own memory built in, the problem ceased. Bad RAM, can, famously, cause file system corruption and thus system rot. Or bad sectors on a hard disk. Could be I've not seen non-hardware-related system rot because I tend to stick pretty carefully to packages provided by my OS's repositories which, in the case of Debain and variants, means almost any piece of software you could want will be available. I tend to shy away from compiling from source, which could be another reason I've not seen system rot and/or performance degradation.

James

[althalus]

Packages installed once and then never removed. Applications that get compiled from source because they aren't in repos. Packages that get compiled from source because the ones in the repos are too old for some other tool which is not in the repo. Sudden unexpected side effects when removing something that should not be in use. Installing something from a PPA repo which creates conflicting versions. These all contribute to bloat and rot.

To say rot and bloat cannot happen on linux is naive. If you stick purely to the packages in your distro's repositories, and do a clean install every new release, then perhaps you can avoid bloat and rot. Sure, it is going to depend upon an individual's usage pattern, but system rot is a very possible problem, especially under ubuntu based systems, where the versions in the repos are often far too old.

TC's principles result in a lean, efficient, quick to boot, operating system which does not needlessly waste resources.

[jur]

Because no Linux system I've set up and used has ever suffered from system rot.

Perhaps you are a bit lucky? Or more careful than the average user? I was ready to give up on linux after one after another problem after problem with ubuntu. And it was rot all right - something would go wrong with the display and rebooting didn't make it go away. I ended up reinstalling ubuntu a number of times and getting very tired of it. Then the new xubuntu ended up not wanting to boot up at all and I had enough - winXP was a lot better than that.

With TCL, it leaves one with total control over your PC. Essentially, it reinstalls with every boot. It was the philosophy that got me hooked on TCL in the first place.

[jamtat]

To say rot and bloat cannot happen on linux is naive. If you stick purely to the packages in your distro's repositories, and do a clean install every new release, then perhaps you can avoid bloat and rot. Sure, it is going to depend upon an individual's usage pattern, but system rot is a very possible problem, especially under ubuntu based systems, where the versions in the repos are often far too old.

Coupla things. One, you're conflating rot and bloat. These two things may be logically related: since all data likely contains at least some corruption, the more data a computer holds, the more corrupt data it is likely to hold. But you're ignoring what I've written if you're saying I naively assume bloat can't happen. Did you read the part of my post where I marvelled at the thousands of discrete files I find on my system? That's a tacit acknowledgement of bloat, in case you didn't understand that. If I was making any point related to bloat it was that, despite the bloat, my system performs just as well as it did the day I installed it--before, theoretically, there was so much bloat. Thus, at least in my case, bloat has no discernable effect on system performance.

Note that I don't argue there are no cases in which bloat doesn't have a detrimental effect on systems. No, having worked extensively with legacy systems, I know well the negative impacts of bloat on those systems. I still recall how, on one of the first systems on which I ever installed Linux (Turbolinux in '98 or '99) how amazed I was that the initial installation took up most of my 1.3 GB hard drive. I also am aware that bloat negatively impacts embedded systems. And in general, I don't like bloat for aesthetic reasons: I generally try to do the most with the least. If saying you're that "system rot" expresses an aesthetic disagreement with the Linux developers/community over how much code should be used to accomplish a given task, or that programs and OS components should be as minimalistic as possible, I'm in full agreement with that sentiment. But, as I said in my OP, I take system rot to mean something else: to me it means the sort of performance degradation that plagued Windows machines (I don't know if XP or Windows 7 are any different) and that would, eventually, require a reinstalation of the OS. And I was arguing that I've never seen that sort of degradation on any Linux system I've set up and used over the last 10 years.

And no, I don't do a clean install every release (Arch has a rolling release schedule, anyway). I haven't done so since I switched to Debian about 6 years ago. I just keep doing a dist-upgrade or pacman -Syu and my machines continue to hum along nicely and effectivley--despite the bloat.

TC's principles result in a lean, efficient, quick to boot, operating system which does not needlessly waste resources.

I'm fully on board with these principles. Sounds great to me. I don't know to what extent these principles will be embraced by the larger community, but I'm already very much in favor of them.  My guess is that, because industry is the major force driving Linux development, and since industry has a vested interest in keeping up the breakneck pace of technological progress (they do it both to out-compete one another, as well as to make greater profits off the new things you need to buy every year to keep up), the TC principles will continue to represent a fringe movement--at least as far as personal computing goes. For embedded systems, on the other hand--and, oddly enough, for legacy system users--TC will likely represent something cutting-edge.

James

[jamtat]

Perhaps you are a bit lucky? Or more careful than the average user? I was ready to give up on linux after one after another problem after problem with ubuntu. And it was rot all right - something would go wrong with the display and rebooting didn't make it go away. I ended up reinstalling ubuntu a number of times and getting very tired of it.

The odds that, over 10 years and countless Linux installations of all flavors, I was lucky in not having a system end up as degraded as Windows systems I've used over longer periods of time, are very, very, low. In fact, they're so low that I discount luck as a factor and attribute the much improved performance I've seen to the resiliency of the OS and to the hard work of the community maintaining it.

It's not to say I haven't had things go wrong. When I used Debian unstable and would do various upgrades, for example, it would happen that something would get broken: I couldn't get a graphical display, the browser would turn flaky, networking would go down, etc. I still encounter problems like that with Arch. They're irritating at the least. But when the solution doesn't require some sleuthing on the internet in order to find out how to tweak some config file or something, it's usually a matter of waiting until the package maintainers fix some issue with the program they packaged and release an update. I don't consider that a "system rot" issue; do you?

So far I've been able to fix each such issue that's arisen, or, after some waiting, to get an updated package to replace the faulty one. I won't say I've never gotten pissed about it; no, I heave a huge sigh whenever this happens, anticipating the trouble-shooting I'll have to get involved in. But I'll also say that I'll be very, very surprised if these sorts of problems don't crop up occassionally on any TC system I may set up. And I'll also be very surprised if maintenance efforts similar to those I've expended so far on my systems will not also need to be made on a TC system.

Has your use of TC really been entirely glitch free so far?

James

[althalus]

Coupla things. One, you're conflating rot and bloat....Thus, at least in my case, bloat has no discernable effect on system performance.

Bloat has quite a negative impact on apt's performance, in my experience. And the reason I so freely mixed bloat and rot, is that my interpretation of TC's core principles is that it aims to solve BOTH problems - and in my experience they are both very tightly coupled problems - system rot really only starts happening on bloated systems.

the TC principles will continue to represent a fringe movement--at least as far as personal computing goes.

Oh, definately. TC will never be as simple to use as the current mainstream distros.

For embedded systems, on the other hand--and, oddly enough, for legacy system users--TC will likely represent something cutting-edge.

TC will be held back in this regard by being x86 - As long as ARM rules embedded land, TC has no place there.