Tiny Core Linux
dCore Import Debian Packages to Mountable SCE extensions => dCore X86 => Topic started by: Jason W on June 02, 2014, 10:16:32 AM
-
With a recompile of the current TC 3.8.13 kernel to include CONFIG_DEVTMPFS=y and a disabling of the zram swap routine in tc-config, I am testing an early working Jessie dCore. There are some quirks with Udev and Xorg, as well as Gnome does not just fire up and run, but other than that things like icewm, XFCE, iceweasel, and my other commonly used apps are working as is. Of course, there are always challenges with any major system-wide upgrade and I am confident that Jessie will make a solid dCore just like Wheezy has become. Jessie will freeze in November, so package versions are still in flux right now but when I get things polished a little more I will post an alpha cut for wider testing.
The main dCore features are pretty much in place and working well it seems. My plans for the coming few months will be to focus on the coming Jessie upgrade as well finding less widely used packages that may need startup scripts or dependencies added.
-
hi jason w,
great news!
actually i wanted to ask you for a dCore-version based on jessie.
usually i use stable and extremely audited software especially based on lean, vanilla, stable debian,
but for some weeks i was fallen almost in love with my jessie-installation on a netbook...
it's stable, modern and there is some fun with fresh packages :)
what do you think about a more recent kernel? ... with the actual i didn't succeed with suspend to ram :(
i'm looking forward to your jessie-cut and will contribute with heavy testing ;)
keep on hacking.
-
Hi netnomad,
When standard Core updates it's kernel is when I adopt it rather than using a different one. That way there is a larger pool of folks using and testing the same kernel rather version than us each having our own separate kernels for x86. Actually, I plan to use the same source, config, and patches and compile it on dCore so that folks can use dCore to compile their own kernel modules if they want to use something outside the standard kernel tree. I have been meaning to do that with wheezy dCore but just did not get around to it and there has not been a request for it yet.
Basically right now the zram swap issue is the holdup, I hope to figure that out soon as I am also eager to get a Jessie dCore up for testing.
-
CONFIG_DEVTMPFS=y
CONFIG_LEGACY_PTYS=y
The above kernel changes fixes the xterm issue and allows most everything to work, but not Gnome, though the Gnome apps work through Icewm. I will figure out Gnome, and as I don't think many use it on Core anyway, it is not a show stopper for an alpha release of the next major version..
These kind of changes are documented out there and are common to whether one is taking on a major update to a source based system or one like dCore. I will continue to polish and hopefully in the coming weeks I will feel it is ready for a public alpha.
-
Actually things have come together more quickly than I anticipated, just an xterm issue to resolve and otherwise all that I use on my normal desktop is just working. I have made some more kernel config adjustments using Gentoo's list of required and recommended config options to support systemd. The good thing is that like Knoppix, we use our own booting and init that is independent of systemd, upstart, etc. So systemd is there but the /etc/init.d scripts work as before and to the user there is no real difference.
So far the startup scripts, dependency lists, prebuilt packages, and data tarballs on the server work across Wheezy, Ubuntu, and Jessie dCore x86 versions. So Jessie dCore will still be under the 5.x major release. Since I plan to stay with one directory to support both existing and new dCore's, at least as long as possible, the next major version can be when enough changes require that we start a new directory of files on the server to cleanly support newer Debian/Ubuntu releases.
Oh, and the packages in Jessie are changing daily, so when testing my advice would be to use the list file feature of importsce and make one large SCE. If you want to add more packages, re-import that SCE with the additions. Otherwise breakage can occur due to conflicting files among SCE's. The xterm issue is all that is left before I feel confident to post a dCore.gz, I aim to fix that tonight.
-
...sounds good.
keep on hacking!!!
-
Thanks netnomad.
Ok, finally I figured out the Xterm issue with users other than tc, just some udev rules changes in base. /lib/udev/rules.d/91-permissions.rules to be exact.
The files are located at:
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/
I will ponder what name or version we wil go with ultimately, but for now dcore-jessie.
When I say if adding packages to re-import one SCE using a file list, I mean it. I broke my system importing separate SCE's in a 24 hour period due to Jessie's constant changes. As we know, re-importing will bring in any changed packages, only downloading what has not been already or what has changed. But that being said, feel free to use, test, and report any issues stating your hardware (laptop, desktop, etc) and whether you are using one large SCE. What I am looking for is needed changes in the kernel, startup scripts, or perhaps base files. My normal desktop is now dCore-jessie, lets have fun with it and we have plenty of time to tweak it before it's November release.
Oh, and xmms requires you choose alsa as the sound output in preferences. I don't remember if that had to be done in Wheezy, but I am all for the "just works" mindset and any supported changes that allow for it in the prebuilt as well as the regular package selection.
-
Glibc has been updated to 2.19 in Jessie, base files in dCore-jessie.gz has been updated and reposted.
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/
-
Posted an update that uses a rotating dash instead of echoing names of dependencies in the terminal during the dependency gathering routine.
-
Updated base libraries and posted a new cut:
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/
-
Updated to handle the creation of /dev/pts and dev/shm which is handled in Debian by initscripts rather than udev.
-
Updated to remove unneeded files in /etc/udev/rules.d/ that were interfering with the creation of entries in /dev/disk that kept Unetbootin from operating.
-
hi jason,
today i tried your last cut.
i booted with base and norestore, but i'm not able to importsce the package mc.
would you give me some hints.
thank you for your help.
-
About 12 hours ago I had a bad cut posted for about an hour. Please download a new dCore-jessie.gz and try again.
Thanks.
Below should be the content of /etc/udev/rules.d:
55-tc.rules
80-drivers.rules
98-tc.rules
90-permissions.rules
-
hi jason,
i guess that the jessie-cut needs that label-adjustments, too.
would you be so kind and check it?
thank you for your great coding.
-
Glad to hear that Wheezy is working now, I had adjusted Jessie at the same time as I was fairly confident it would be a go. Have you tried it yet?
-
hi jason,
with jessie i have no luck with your last cut.
by the way, i use the backup-file and the sce-packages from my wheezy setup.
-
Don't try to mix wheezy and jessie sce's, it will not work.
By no luck, you mean the LABEL= boot option does not find the right device, or that there are issues with the packages?
-
hi jason,
in my opinion my central problem is still the label-issue...
the error messages are the same as i experienced with wheezy.
sce-package-error-messages caused by a conflict between wheezy and jessie look different.
-
hi jason,
importsce works fine,
but my tce-directory and my backup is not found during the boot process,
although i can manually mount my second partition on my usbstick that provides them.
if i avoid the label-option, the bootcodes restore=sdb2 and tce=sdb2/tce don't work, too :(
-
Ok, I will look into it soon.
-
Netnomad - pardon the delay, I have been busy lately. Via ssh login to my box at home I think I see the simple culprit and I will fix and test it when I get home tonight.
-
hi jason,
sounds great!
thank you for your work and commitment.
-
I think I have fixed it, uploaded:
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz.md5.txt
-
hi jason,
with this cut i succeed without label:
kernel /live/dcore-testing/boot/vmlinuz waitusb=5 protect restore=sdb2 tce=sdb2/tce lst=sceboottest.lst vga=795 psmouse.proto=imps settime tz=CET-1CEST,M3.5.0,M10.5.0/3 laptop kmap=de multivt lang=de_DE.UTF-8 mydata=dcoret blacklist=snd_usb_audio intel_rng.no_fwh_detect=-1 printk.time=1 cron desktop=fluxbox log=3 acpi_sleep=s3_bios,s3_mode showapps
append initrd=/live/dcore-testing/boot/dCore.gz
but there are still problems with labels, perhaps its just the waitusb-command?
kernel /live/dcore-testing/boot/vmlinuz waitusb=5:LABEL=tc protect restore=LABEL=tc tce=LABEL=tc/tce lst=sceboottest.lst vga=795 psmouse.proto=imps settime tz=CET-1CEST,M3.5.0,M10.5.0/3 laptop kmap=de multivt lang=de_DE.UTF-8 mydata=dcoret blacklist=snd_usb_audio intel_rng.no_fwh_detect=-1 printk.time=1 cron desktop=fluxbox log=3 acpi_sleep=s3_bios,s3_mode showapps
append initrd=/live/dcore-testing/boot/dCore.gz
-
hi jason,
i guess that waitusb has problems with labels!
waitusb=10:LABEL=tc
makes problems :(
but
waitusb=5
works fine :)
-
hi jason,
in my configuration the swap based on zram does not work and in wbar the xterm icon does not appear :( ,
but the rest looks pretty promising :)
keep on hacking!
-
Zramswap is still a work in progress. I will look further into the rest.
-
I can boot here with waitusb with both the UUID and LABEL.
waitusb=30:LABEL=SCE
waitusb=30:UUID=EF2B-4EE5
It echoes that it is waiting, then after a second or two it continues booting once the drive has been awakened.
-
I have confirmed that Wheezy dCore is working as expected with the label and uuid specific waitusb commands as well as the uuid and label specified tce directory.
I will make these changes in the Ubuntu dCores as well.
-
hi jason,
for me waitusb with label is still unstable, but i have no problem to change my bootcode to waitusb=5 without label.
wheezy and all other cuts work fine with waitusb=5:LABEL=tc.
never mind, thank you for your help and commitment.
-
What is the md5sum of your dCore-jessie.gz?
Just to be sure.
And do you see the "Waiting as requested.." line when booting dCore-jessie at all?
And what is the output of the below command once dCore-jessie has booted:
blkid -it LABEL=tc
-
What is the md5sum of your dCore-jessie.gz?
i renamed dCore-jessie.gz to dCore.gz:
tc@box:~/.MNT/sdb1/live/dcore-testing/boot$ md5sum dCore.gz
1b2e5d9ba7ff976652f48698a3a44664 dCore.gz
that md5sum should be correct.
blkid -it LABEL=tc
tc@box:~/.MNT/sdb1/live/dcore-testing/boot$ blkid -it LABEL=tc
The low-level probing mode requires a device
tc@box:~$ blkid -s LABEL /dev/sdb2
/dev/sdb2: LABEL="tc"
tc@box:~$ sudo lsblk -o name,label,mountpoint /dev/sdb
NAME LABEL MOUNTPOINT
sdb
|-sdb1 MYSTICK /mnt/sdb1
`-sdb2 tc /mnt/sdb2
i attached the photos of my two boot-versions,
first with label the failed boot process :(
second without label and a successful boot process :)
-
The command is "blkid -lt LABEL=TC", I made a typo.
Oh, and I assume you have renamed vmlinuz-jessie to vmlinuz?
One thing that may help is to start with only the tce directory being specified on the boot line, and if that works start adding each one back one at a time and see at what point it fails.
-
The command is "blkid -lt LABEL=TC", I made a typo.
tc@box:~$ blkid -lt LABEL=TC
tc@box:~$ sudo blkid -lt LABEL=TC
tc@box:~$
there is no echo or result to this command.
Oh, and I assume you have renamed vmlinuz-jessie to vmlinuz?
yes, you're right.
One thing that may help is to start with only the tce directory being specified on the boot line, and if that works start adding each one back one at a time and see at what point it fails.
my whole bootcode works fine, only waitusb with label-mode waitusb=5:LABEL=tc makes problems:
kernel /live/dcore-testing/boot/vmlinuz waitusb=5 protect restore=LABEL=tc tce=LABEL=tc/tce lst=sceboottest.lst vga=795 psmouse.proto=imps settime tz=CET-1CEST,M3.5.0,M10.5.0/3 laptop kmap=de multivt lang=de_DE.UTF-8 mydata=dcoret blacklist=snd_usb_audio intel_rng.no_fwh_detect=-1 printk.time=1 cron desktop=fluxbox log=3 acpi_sleep=s3_bios,s3_mode showapps
append initrd=/live/dcore-testing/boot/dCore.gz
-
Using lowercase tc with the command should work. If not, then that is the issue as the blkid command is used to check whether the device is available and then exists the waitusb loop.
-
tc@box:~$ blkid -lt LABEL=tc
/dev/sdb2: LABEL="tc" UUID="6cc7058b-ac89-40a1-8700-e43e47539ce3" SEC_TYPE="ext2" TYPE="ext3"
in my common bootcode i use lowercase, too:
waitusb=5:LABEL=tc
-
Ok, I will try my setup with a two digit label and ext3, now I have fat32 with label SCE. It shouldn't make a difference, but there is something making my setup work and yours not.
-
tce=LABEL=tc/tce
Why not
tce=LABEL=tc
?
-
hi tinypoodle,
you are right that there is no need for
tce=LABEL=tc/tce
so i tried also
tce=LABEL=tc
there is no difference for me, i have tried both.
the faq describes the option to use a path to the tce-directory, f.e. tce=LABEL=tc/path/tce,
so both should work.
-
I have found that there seems to be an issue when using an ext2 formatted usb drive.
I will look into it.
-
This was a udev issue in Debian Jessie that needed a simple udev fix. Thank you netnomad, JLS, and any others that tested and reported this issue, for helping to make dCore/dCore-jessie better than it was yesterday. The fix is also applicable or at least not damaging to dCore Wheezy, also updated in the release_candidates area.
-
hi jason,
yeah, you solved it :)
thank you for your great work.
-
hi jason,
sorry, but there are still minor issues, so the last cut is still a bit unstable.
i guess a warm reboot of dcore-jessie could initiate errors,
rebooting from tinycore 4.7.7 to dcore-jessie works better.
a cold boot process seems to be more stable.
there could be a problem with the reboot of unclean-ext3-filesystems, too?
or is it just the problem that a warm reboot causes issues with the init of the usb-bus?
you improved the situation, but some reboots still fail :(
-
Does your usb drive have an unclean file system? Fixing the fs would remove one variable that could certainly cause issue.
What errors do you see?
EDIT: I am still having LABEL boot issues with ext2 usb, I will look into it.
-
hi jason,
no, my usbstick and it`s filesystems seems to be flawless,
there are no error messages,
but nevertheless i check it regulary with
sudo e2fsck -pv /dev/sdc2
i just asume that perhaps dcore-jessie could have problems with the shutdown:
that the usbstick is not cleanly unmounted or has problems to be reinitialized and mounted at the next reboot?
... i just try to find reasons for the problem.
by the way the usbstick makes no problems with other cuts of tinycore or other livessystems!
-
Thanks netnomad for testing. I still see an issue with my own ext2 usb bootable drive and will find a fix. The sytstemd thing in Jessie has provided a set of challenges that will likely not be seen the next Debian version upgrade.
But I am also of the mind that I would rather make one dCore that is excellent than have several that are in various stages of repair, which is the case of packages between the dCores available. Debian stable does not upgrade libraries or apps except for bugfixes in its lifecycle, and Ubuntu seems to upgrade more often which can cause breakage. So a Debian stable dCore as the official dCore is much more doable as there are so many packages and basically one dCore maintainer. The next testing version of dCore will be available when things settle down in Debian testing, I plan it next time to be more closer to the next Debian release so I can do this kind of bugfixing before there is a public dCore testing release. The Ubuntu dCores were a test of concept, and it would work with Linux Mint, Kali, etc. But again, better to support one good implementation than spread development effort across a number of concepts.
So for right now, "waitusb=5" or similar will do, and I will work on a fix that allows the LABEL to be specified in the waitusb command to maximize performance.
-
hi jason,
thank you for sharing your plans and concepts.
keep on hacking :)
-
Updated the release version info to 5.14.9.27. Please report any issues.
-
...works flawless, smooth and fine for me :)
-
Updated to deal better with groups the user needs to belong to and that would require a re-login to effect upon adding the user to group. Please download and test.
-
Updated. Main fix is a hopefully error proof way of dealing with the waitusb boot code when specifying UUID or LABEL. In Jessie and Trusty, blkid being able to read the label/uuid of a usb device does not mean the device is ready to be mounted, nor does a dry run of mount issue the right exit code upon failure. Actually being able to mount the device is what is tested in the routine.
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz.md5.txt
-
Updated for the issues found in dCore-wheezy:
Added the ability to call unloaded ondemand apps from a root terminal if root use is desired like in gparted. Also fixed the issue of ondemand not obeying command arguements.
I added an exit code of 1 in debExtract where tar and ar are used in the case of their failure. Deb2sce was supposed to exit on a fail of debExtract, but the tar and ar commands failing were not causing the debExtract script to exit with a 1. Hopefully it is more robust now, the md5sums of the debian packages and prebuilt were already being checked whether being redownloaded or using what is already in the sce dir, my hunch is normally an out of disk space issue if there is failure.
Please download and test.
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz.md5.txt
-
hi jason,
is there an issue with the md5sum?
did you include the scepkglist-script in this cut?
thank you for your help.
-
With what md5sum? Please give more details. Do you mean why make more checking for the md5 checking already in importsce? That is a good question, but if a "tar" or "ar" function fails, then it does not automatically cause a failure of debExtract. There was added some testing of the ar and tar functions in debExtract that would make a failing of tar and ar a failing of debExtract, which would make importsce (via deb2sce) more solid.
-
hi jason,
sorry for my misleading, imprecise message!
i mean the md5sum-file for the last cut of dCore-jessie.gz
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz.md5.txt
bd3a30f6a2f43e90433da65b2c475445 dCore-jessie.gz
then the md5sum of the just downloaded file from that url:
tc@box:~/tmp$ md5sum dCore-jessie.gz
dde5dc98bd3977099ca8310d38bd64d9 dCore-jessie.gz
-
Oops, I had left the md5sum file in place from the unpack of the previous cut as a filename marker and forgot to remake it. Should be good now.
-
Updated to change the "importsce -f" option to the more logically named "importsce -l". Also, a /etc/sysconfig/tcedir/sce/pkgname.sce.lst file is created out of that specified file list. That .lst list will be used on future import sessions of that package name whether a Debian one or one of the user's choosing. Using the "importsce -l" option on future imports of that package name with an expanded package list will create a new sce with those packages and also overwrite the previous .lst list file.
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz.md5.txt
-
Fkxed a warning message that occurs when the list file option is not used, uploaded new cut.
-
hi jason,
i didn't understand the concept of the *.lst-files yet.
so
1. i need a file with the packagelist, that's named f.e. mypackage
2. the command "importsce -l mypackage" copies the file to mypackage.lst
3. mypackage.lst is used for the import-process
i guess that usually mypackage and mypackage.lst are identical, what's the idea behind that?
mypackage is needed for the command, mypackage.lst is needed for the import-process?
thank you for your support and help.
-
Here is how it works.
importsce -l mypackage will create an sce out of the pakcage list named mypakcage.
That list is then stored as /etc/sysconfig/tcedir/sce/mypackage.sce.lst.
/etc/sysconfig/tcedir/sce/mypackage.sce.lst will be used as the package list for future re-imports of mypackage,
unless a new list named mypackage is used, which in that case the new list is used for importing and that
new list is copied to /etc/sysconfig/tcedir/sce/mypackage.sce.lst.
You can either pre-create /etc/sysconfig/tcedir/sce/mypackage.sce.lst or just use the "importsce -l" option as before. This way you don't have to try to remember where you stored that package list or if you have added or taken from it since you last imported using that list. The .lst file is automatically used in re-imports, allowing easy updates to meta extensions with the same contents as the previous import.
-
In fact, this combined with the non-interactive feature will allow a simple scripted update of the existing sce's in the sce directory with one command similar to a Core extension update. That is, assuming the sce's made with a package list have their .lst file in place, otherwise the import will fail or simply import the Debian package and it's deps if the sce name is that of a Debian package.
-
Updated to release_candidates with a fix for non-interactive mode on package selection.
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz.md5.txt
-
Updated to remove pathname from command line when present during importsce.
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz.md5.txt
-
Updated to make import sessoins more time efficient and to create the tool updatesce.
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz.md5.txt
-
hi jason,
updatesce breaks after the missing md5sum of firmware-all.tar.gz:
Fetching updated firmware-all.tar.gz..
converted 'http://repo.tinycorelinux.net/5.x/x86/import/firmware-all.tar.gz' (ANSI_X3.4-1968) -> 'http://repo.tinycorelinux.net/5.x/x86/import/firmware-all.tar.gz' (UTF-8)
--2014-11-13 19:07:00-- http://repo.tinycorelinux.net/5.x/x86/import/firmware-all.tar.gz
Resolving repo.tinycorelinux.net (repo.tinycorelinux.net)... 89.22.99.37
Connecting to repo.tinycorelinux.net (repo.tinycorelinux.net)|89.22.99.37|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2014-11-13 19:07:01 ERROR 404: Not Found.
md5sum: firmware-all.tar.gz.md5.txt: no properly formatted MD5 checksum lines found
md5sum failed for firmware-all.tar.gz, exiting..
-
Ok, a simple issue in deb2sce when GNU wget is installed. I aliased wget and md5sum to their busybox versions in /bb. Uploading now, redownload and try again.
-
Updated with simple bugfixes and enhancements.
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz.md5.txt
-
Uploaded with bugfixes releted to updatesce.
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz
http://tinycorelinux.net/5.x/x86/release_candidates/dCore-jessie/dCore-jessie.gz.md5.txt
-
Debian Jessie has been frozen as of Nov 5, meaning that any future package updates are for bug fixes only. To dCore this means that Jessie can be considered stable for daily use and normal importsce methods. No longer is one single SCE from a file list the only recommended way.
When the latest changes have settled down in importsce, I will make a stable release of Jessie, as well as Trusty so users don't have to be subject to the testing changes unless they desire to.
-
Updated with latest fixes. Available in the release_candidates directory below. Note, the releases and release candidates under the tinycorelinux/5.x/x86/ directory will no longer be supported once a stable release is make in tinycorelinux/dCore/x86/. Users are advised to download the latest release at that time.
http://tinycorelinux.net/dCore/x86/release_candidates/dCore-jessie/
-
A smalll update with more streamlined md5sum and dependency searching during import.
http://tinycorelinux.net/dCore/x86/release_candidates/dCore-jessie/
-
hi jason,
a great cut!!!
i can confirm you after pretty heavy testing that i found no flaws or bugs :)
thank you for your efforts.
-
Great. But I have uploaded one more bugfix for the dCores dealing with the md5sum checking of packages in extra repos.
Please download and test.
-
Bugfix in extra repo md5sum function.
http://tinycorelinux.net/dCore/x86/release_candidates/dCore-jessie/
-
Bugfix.
http://tinycorelinux.net/dCore/x86/release_candidates/dCore-jessie/
-
hi jason,
very stable cuts, congratulations!
thank you for your work.
-
Updated. Import creates an md5.txt file for created SCE's, and loadsce checks for it's existence and it's md5 if it exists. This won't break existing SCE directories, it will simply add an md5.txt file for future imported SCE's.
http://tinycorelinux.net/dCore/x86/release_candidates/dCore-jessie/
-
hi jason,
this cut produces no error messages,
but nevertheless some packages are not merged, are missing after an import without errors.
my base-package works flawless,
###
000-base-jessie:
console-setup
console-data
bash
mc
ssh
rsync
screen
grep
tar
zip
unzip
gzip
bzip2
xz-utils
p7zip-full
nmap
iptables
fail2ban
redsocks
pdnsd
###
but my based-on-package lacks of icedove and some more packages (the size is 50mb smaller)
###
100-desktop-jessie:
xorg-all
Xprogs
wbar
fluxbox
iceweasel
icedove
evince
jpilot
xvnc4viewer
###
-
Ok, I see the issue. Grep only allows you to search a fixed number of lines and not until the next line like sed or awk. Both "original awk" in Debian as well as Busybox awk are slow but accurate in searching the Packages file until the next line. Gnu grep is the fastest tool, but as there are going to be new fields in the Packages file as well as some I see now I have not yet specified to be excluded (there are many) then grep is not going to be reliable for this task.
But sed is reliable and takes about a tenth of a second per dependency to produce it's dependency list. Awk takes about a second, so sed is still very much a speed improvement but will not result in error if there are new or overlooked fields in the Packages file now or in the future.
I will work it in to the md5 and dependency functions, may take a few days.
-
thank you for your reply :)
-
Grep and sed are not dealing with deleting parentheses and their contents which are often found in extra repos. Sed works as it should for this function on the command line but for some reason snippets are getting through and are being listed as package names and stopping the dependency gathering routine in mid stream.
But using sed to parse the needed info from the main debinx packages file into it's own file is fast and 100% accurate so far, and then using the awk command on that file is also fast and 100% accurate. I hope to finish testing tonight and will post the results soon.
-
Updated, please test.
Import and load the "sed" package for better performance during import.
http://tinycorelinux.net/dCore/x86/release_candidates/dCore-jessie/
-
hi jason,
the last cut produces still a broken package for me :(
twice i ended up to the same result: around 7mb are missing...
tc@box:~/.TCE/sce/update$ ls -la 000-base-jessie.sce
-rw-r--r-- 1 root root 90062848 Nov 27 14:56 000-base-jessie.sce
tc@box:~/.TCE/sce$ ls -la 000-base-jessie.sce
-rw-rw-r-- 1 tc staff 97030144 Nov 23 21:45 000-base-jessie.sce
one thing is interesting:
my 000-base-jessie worked fine in the last cuts, i had only problems with my 100-desktop-jessie-package.
thank you for all your efforts.
-
Is you 000-base-jessie list the same as before?
I am getting this size when I import that list now.
me@box:/mnt/sda3/tceimport-jessie/sce$ ls -l 000-base-jessie.sce
-rw-rw-r-- 1 peter staff 50831360 Nov 27 18:50 000-base-jessie.sce
me@box:/mnt/sda3/tceimport-jessie/sce$
-
hi jason,
you are right, i didn't changed it:
tc@box:~/.TC/tce/sce$ cat 000-base-jessie.sce.lst
console-setup
console-data
bash
mc
ssh
rsync
screen
grep
tar
zip
unzip
gzip
bzip2
xz-utils
p7zip-full
nmap
iptables
fail2ban
redsocks
pdnsd
tcpdump
gnupg
laptop-mode-tools
pm-utils
man-db
ntfs-3g
lvm2
cryptsetup
firmware-all
wifi
wireless
wpasupplicant
checkmissingdebs
importupdatecheck
libsmartcols1
-
Ok, I see the same difference, 97mb versus 90.
This has been a massive set of changes recently as far as code, hopefully in the coming days I will fix what we are seeing lately.
-
Ok, I see the same difference, 97mb versus 90.
hi jason,
good news that you can reproduce this error.
keep on hacking, we appreciate your good work :)
-
Updated, reverting to using grep for the md5 and dependency routines, sed no longer used. Turns out I went around my elbow to get to my ear on that one, though grep itself can't search until next new line, a simple grep trick can.
http://tinycorelinux.net/dCore/x86/release_candidates/dCore-jessie/
-
hi jason,
the last cut looks very promising, stable and flawless for me :)
thank you for your contributions.
-
This thread is now closed in favor of the Release and Release Candidate sections above. Please direct any posts there.