Tiny Core Linux

Tiny Core Extensions => TCE Talk => Topic started by: MTCAT on April 16, 2021, 04:32:18 PM

Title: [Solved] TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 16, 2021, 04:32:18 PM
Hi everyone,

With an amazing amount of help, almost all from Rich, I've been able to properly install TC3_8.4 (2.6.33.3), build and install an ADC driver for my fancy A-D card, and have it auto-load at boot time, installed a nice little clock, and most recently, got gpsd and chrony working to synchronize system time to GPS NMEA and PPS sources to ~ 1 us.

Unfortunately though, for my data-acquisition program to work properly, I need to use a real-time kernel.

I downloaded the real-time patch from;

https://mirrors.edge.kernel.org/pub/linux/kernel/projects/rt/2.6.33/

I think I should leave my existing (very nice) install alone, and start all over again for the real time case, with a new thumb drive ?

To start with, I'll follow the same procedure Rich sent me before (see attached), booting up TC from one pen drive, and installing to a new fresh pen drive.

After that, I'm pretty much in the dark, I need to apply the patch and "re-master" modules ?

Is the general process as outlined by Rich on page 2 of the following post ?

http://forum.tinycorelinux.net/index.php/topic,23797.0.html

After I can hopefully get the rt patch applied, I can re-install the ADC driver, gpsd, chrony, etc, and then I can finally try to get the data acquisition program going !

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 16, 2021, 06:14:17 PM
Hi MTCAT
... I think I should leave my existing (very nice) install alone, and start all over again for the real time case, with a new thumb drive ? ...
That's what I would do.

Quote
... Is the general process as outlined by Rich on page 2 of the following post ? ...
If you mean this one, yes:
http://forum.tinycorelinux.net/index.php/topic,23797.msg149402.html#msg149402
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 17, 2021, 08:00:05 AM
Thanks Rich, I'll start going on this today hopefully !
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 17, 2021, 11:13:44 AM
Hi Rich,

Would this be the right path to the standard 2.6.33.3 config file ?

http://www.tinycorelinux.net/3.x/release/src/kernel/config-2.6.33.3-tinycore

First step in your big process is to "wget" the original config file.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 17, 2021, 02:00:19 PM
Hi MTCAT
Yes, that's the correct config file.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 17, 2021, 02:45:07 PM
Hi Rich,

So far so good,  I just ran the "make -j 2 bzImage" command and all is well (I think), but, the next command in your list is;

cp arch/x86/boot/bzImage  /mnt/sdg1/Linux/vmlinuzASUS64rev1

In my case, I'm installing to sdb1 but, I don't have the directory structure "/mnt/sdb1/Linux/.." In /mnt/sdb1 I have the following directories
"./"
"../"
"boot"
"lost+found"
"tce"

Do I need to copy bzImage to one of these directories ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 17, 2021, 05:30:54 PM
Hi MTCAT
Do this:
Code: [Select]
cp arch/x86/boot/bzImage  /mnt/sdb1/boot/vmlinuzRT1This way you don't overwrite the kernel that's already there.

If you look at  /mnt/sdg1/boot/extlinux/extlinux.conf  you should find something similar to this:
Code: [Select]
DEFAULT core
LABEL core
KERNEL /boot/vmlinuz
APPEND initrd=/boot/core.gz quiet home=UUID="9764a623-4226-4cc2-b777-35f1c7446202" opt=UUID="9764a623-4226-4cc2-b777-35f1c7446202" waitusb=5:UUID="9764a623-4226-4cc2-b777-35f1c7446202" tce=UUID="9764a623-4226-4cc2-b777-35f1c7446202"

Change it to this:
See attached file.

This should boot to your real time kernel but still allow you to boot to the stock kernel if you have a problem.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 17, 2021, 06:48:08 PM
Hi Rich,

Thanks a lot, that sounds good (to be able to boot either one), does it matter if I keep the "bzImage" naming convention though ?

My original kernel is called "bzImage", does it matter if I call the real-time kernel "bzImageRT1" instead of "vmlinuzRT1" ?

Also, will I need a new initrd file for the real time as well ? My original is called "tinycore.gz", so the real time could be called "tinycore-rt.gz" maybe ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 17, 2021, 07:05:46 PM
Hi MTCAT
I forgot they still called it bzimage in the older kernels. You can call it anything you want. Just make sure you use the
same name in you extlinux.conf file.

... Also, will I need a new initrd file for the real time as well ? ...
I don't know. That depends on what the real time patch changed. Go with this one for now.

Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 18, 2021, 11:07:17 AM
Hi Rich,

I ran the sorter.sh script, (but I had to get it from http://tinycorelinux.net/3.x/release/src/kernel/sorter,sh-----> the github link didn't work for me---> I got this sorter.sh with "wget", and then did a "chmod 775 sorter.sh" and then ran it--->seemed to be okay), so that was the last thing I did in your big list (I didn't make an initrd or anything), on power down I made a backup, and now tried to re-boot.

On boot-up, I wasn't given a choice between the old kernel and the real-time kernel, I just saw a little "boot: " prompt echoed to screen, and then eventually, off it went with the real-time kernel.

But, during boot-up, I did see an error,

modprobe: chdir (2.6.33.3-rt19-tinycore): No such file or directory
Failed to open /dev/ramzswap0: No such file or directory
swapon: can't stat '/dev/ramzswap0': No such file or directory

And then the boot-up process continued momentarily, but then hung-up on "Loading Extensions", by hung up, I mean it just gets stuck there, and doesn't go any further.

Any ideas what I did wrong ? I used the "bzImageRT1" naming convention for the kernel and am using the "regular" initrd (tinycore.gz) for both the "regular" and "real-time" kernels.

Thanks,

David

Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 18, 2021, 08:34:40 PM
Hi MTCAT
... modprobe: chdir (2.6.33.3-rt19-tinycore): No such file or directory ...
I think you need to rebuild the initrd. Your kernel version  string changed from  2.6.33.3-tinycore  to  2.6.33.3-rt19-tinycore.

The sorter script created a bunch of kernel related extensions (filesystems-$KERNEL.tcz, alsa-modules-$KERNEL, etc.).
The $KERNEL part of those extension names should be 2.6.33.3-rt19-tinycore. Copy all of them to your  tce/optional  directory.
Code: [Select]
cp *-2.6.33.3-rt19-tinycore.tcz /mnt/sdb1/tce/optional
The sorter script also created base_modules.tgz. Copy that file into your home directory (/home/tc/).
Then continue with the instructions in the attached file. Pay close attention to syntax, especially those single periods like
the one after the  find  command and near the  depmod  command.

When you've finished with the attached file, you will have a  tinycoreRT1.gz  file in your  boot  directory. Make sure you
adjust the real time entry in your  extlinux.conf  file to point to  tinycoreRT1.gz.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 19, 2021, 01:10:34 PM
Hi Rich,

Do you know where all the *-2.6.33.3-rt19-tinycore.tcz files will be located ? And the base_modules.tgz too ?

I made a directory in home called linux-2.6.33.3 or something like that, and the new real-time kernel got built in there.

Would it be possible to do all this by booting up my old non-RT kernel, and then modifying the files on the pen drive which contains the RT kernel ? Or, should I just not be lazy and start all over, right from scratch, install TC3.8.4, apply the patch, build the kernel, make the modules (which took forever !), etc.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 19, 2021, 02:36:57 PM
Hi MTCAT
Do you know where all the *-2.6.33.3-rt19-tinycore.tcz files will be located ? And the base_modules.tgz too ? ...
They should wind up in the same directory as the  sorter.sh  script.

Quote
... Would it be possible to do all this by booting up my old non-RT kernel, and then modifying the files on the pen drive which contains the RT kernel ? ...
I don't see why not. You may have to modify some of the instructions in the attachment. The  /mnt/sdb1/  parts come
to mind. Those should be adjusted to point to the drive with the RT kernel.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 20, 2021, 01:35:44 PM
Hi Rich,

Fired up the VDX3 running off the old non-RT kernel (sdb1), I plugged in the pen drive with the RT kernel on it, mounted it as sdc1, cd to /mnt/sdc1 and I don't have home directory there ! I guess extlinux is not set up yet for persistent home and opt ? I did make a backup though with the RT kernel (on sdc1) when I powered down some time ago, there's a big file called "mydata.tgz" in /mnt/sdc1/tce, I assume that's the backup.

What do you think, could we make use of the backup, or maybe I should just start over from scratch ? It wasn't too bad except the modules took quite long to build, couple hours maybe.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 20, 2021, 01:53:32 PM
Hi MTCAT
Edit the  extlinux.conf  file on the  RT  thumb drive and change  DEFAULT  back to  core.
Boot the  RT  thumb drive.
Follow the instructions for persistent /home and /opt directories here like you did last time:
http://forum.tinycorelinux.net/index.php/topic,24813.msg157808.html#msg157808
Then follow the instructions in Reply #39:
http://forum.tinycorelinux.net/index.php/topic,24813.msg157840.html#msg157840

Make the changes to both sections of the  extlinux.conf  file (core  and  realtime).
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 20, 2021, 03:14:03 PM
Hi Rich,

Thanks for all the help, I did as instructed in your last post, and that all went okay, and then I went ahead and copied all the kernel files (if that's the right word) to /mnt/sdb1/tce/optional, and then I copied the base_modules.tgz to /home/tc and then started going through the instructions in the file you attached (to build a new initrd).

I got up to the second move command, "sudo mv ../base/usr/local........" and that didn't work for me, I have only a "lib" directory underneath "base", there's no "usr" directory under "base" at all ?

Not sure what I did wrong.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 20, 2021, 05:06:31 PM
Hi MTCAT
I don't think you did anything wrong. I think that version of the sorter script doesn't create that path. Replace the second
mv  command with:
Code: [Select]
sudo mkdir -p usr/local/lib/modules/2.6.33.3-rt19-tinycore/kernel/
sudo ln -s /usr/local/lib/modules/2.6.33.3-rt19-tinycore/kernel/ lib/modules/2.6.33.3-rt19-tinycore/kernel.tclocal
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 20, 2021, 06:03:37 PM
Hi Rich,

After I ran the "sudo ln -s..." command, there was echoed to screen;

ln: lib/modules/2.6.33.3-rt19-tinycore/kernel.tclocal: File exists

Is that okay ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 20, 2021, 06:38:00 PM
Hi MTCAT
Do this:
Code: [Select]
ls -l lib/modules/2.6.33.3-rt19-tinycore/kernel.tclocalIt should point to  /usr/local/lib/modules/2.6.33.3-rt19-tinycore/kernel/. The response should look similar to this:
Code: [Select]
lrwxrwxrwx 1 root root 47 Feb 14 16:30 lib/modules/2.6.33.3-rt19-tinycore/kernel.tclocal -> /usr/local/lib/modules/2.6.33.3-rt19-tinycore/kernel/You only care about the paths on either side of the  ->  symbol.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 20, 2021, 06:47:16 PM
Hi Rich,

Yup, when I execute the ls -l command, I get the same response except mine returns

"lrwxrwxrwx 1 tc staff ......"  instead of  "lrwxrwxrwx 1 root root ......"

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 20, 2021, 06:50:22 PM
Hi MTCAT
That should be fine.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 20, 2021, 06:58:29 PM
Hi Rich,

Great, so I should be okay to move on to the next command as shown below ?

sudo depmod -a -b . 2.6.33.3-rt19-tinycore

I'm doing all this in "tempdir", I notice that there's another "cd tempdir" command immediately following the above, yet I never left "tempdir" so, hopefully that's okay as well.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 20, 2021, 07:14:04 PM
Hi MTCAT
Yes, continue with  depmod.

... I'm doing all this in "tempdir", I notice that there's another "cd tempdir" command ...
That was just to emphasize you need to be in that directory when repacking the initrd.

I copied it from one of my other posts:
http://forum.tinycorelinux.net/index.php/topic,24542.msg155744.html#msg155744
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 20, 2021, 07:35:08 PM
Hi Rich,

Darn-it, I ran the "sudo depmod -a -b . 2.6.33.3-rt19-tinycore" command and got the following error;

depmod: ./kernel.tclocal: No such file or directory

For the modules, check out the attached file, which I did in /home/linux-2.6.33.3, anything in there jump out at you ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 20, 2021, 07:54:47 PM
Hi MTCAT
If you do this:
Code: [Select]
ls -l lib/modules/2.6.33.3-rt19-tinycore/Do you see  modules.alias  and  modules.dep  files?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 20, 2021, 08:01:21 PM
Hi Rich,

Yes I do, there's a modules.symbols as well, I also see those "source" and "build" modules I tried to remove before (but didn't), and a "kernel" directory, and "kernel.tclocal" file.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 20, 2021, 08:09:14 PM
Hi MTCAT
... I also see those "source" and "build" modules I tried to remove before ...
You meant directories, right? Remove them:
Code: [Select]
sudo rm -f lib/modules/2.6.33.3-rt19-tinycore/source
sudo rm -f lib/modules/2.6.33.3-rt19-tinycore/build
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 20, 2021, 08:14:55 PM
Hi Rich,

The "source" and "build" are one of those symbolic link things (I think), this is what it looks like for those items after doing the ls -l lib/modules/2.6.33.3-rt19-tinycore/ command.

lrwxrwxrwx 1 tc staff 23 Apr 20 21:52 build -> /home/tc/linux-2.6.33.3/
lrwxrwxrwx 1 tc staff 23 Apr 20 21:52 source -> /home/tc/linux-2.6.33.3/

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 20, 2021, 08:18:32 PM
Hi MTCAT
Yes, they point back to the source directory. The links are not needed. I don't think leaving them will hurt anything either.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 21, 2021, 10:56:37 AM
Hi Rich,

Are there any other things we can try to get around the error seen in reply 23 ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 21, 2021, 11:40:45 AM
Hi MTCAT
I don't think you need to. The  modules.alias  and  modules.dep  files are present. The location that link  kernel.tclocal
points points to doesn't exist. When you boot the RT kernel, then it will exist. Until you load an extension containing
kernel modules, it won't be populated. That's my long winded way of saying I think you can ignore it.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 21, 2021, 04:33:35 PM
Hi Rich,

Oh, okay, that sounds good, so can I just skip ahead to the last step to build the new initrd ?

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 21, 2021, 05:35:51 PM
Hi MTCAT
Yes.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 22, 2021, 06:41:28 PM
Hi Rich,

I tried to run the last command ("sudo find . | sudo cpio.......) but got an error, permission denied trying to create /mnt/sdb1/boot/tinycoreRT1.gz

I'm not sure how we're making a new initrd anyway when we're not adding any new modules really (since all the prior commands failed ) ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 22, 2021, 06:59:28 PM
Hi MTCAT
When you executed this:
Code: [Select]
sudo rm -rf lib/modules/2.6.33.3-tinycore
sudo mv ../base/lib/modules/2.6.33.3-rt19-tinycore lib/modules/
you replaced the old modules with the RT modules that belong in the initrd.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 22, 2021, 07:18:48 PM
Hi Rich,

Oh, okay, thanks, I forgot about which commands worked and which didn't.

Am I maybe missing a "sudo" somewhere in the last command to create the new initrd ?

Thanks,

David

 
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 22, 2021, 07:47:57 PM
Hi MTCAT
... Am I maybe missing a "sudo" somewhere in the last command to create the new initrd ?
The command is in the attachment here:
http://forum.tinycorelinux.net/index.php/topic,24944.msg158827.html#msg158827

Did you remember to:
Code: [Select]
cd tempdir
Is the thumb drive mounted as  sdb1 ? Does the destination exist:
Code: [Select]
ls -l /mnt/sdb1/boot/
Title: Re: TC3.8_4 Real Time Upgrade
Post by: patrikg on April 22, 2021, 10:03:54 PM
A tip.

When I do sudo with piping, i do it this way, it always work.

sudo sh -c 'find . | cpio .....'
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 23, 2021, 07:35:15 AM
Hi Rich and patrikg,

I typed in the command correctly (pretty sure), and I was in tempdir, and sdb1 was mounted, so not sure, I will try again this morning just to make sure.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 23, 2021, 07:47:29 AM
Hi MTCAT
Depending on what else is connected to the computer, there's no guarantee that your device always shows up as sdb1.
The  ls  command can confirm that.

If you confirmed everything else is correct and you still get permission denied, you can try it like this:
Code: [Select]
sudo su
Place the  find . | cpio ....  command here without the  sudo  parts.
exit
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 23, 2021, 07:57:08 AM
Hi Rich,

Changing user to "root" and executing the command worked !, thanks....however, the tinycoreRT1.gz file that got made is very small, 554 bytes ??, whereas the original tinycore.gz is 8085858 bytes ?? Does that seem odd to you ? Sorry for all the problems. Also tinycore.gz is shown as "--r--r--r--" whereas tinycoreRT1.gz is shown as "-rw-r--r--".

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 23, 2021, 08:09:44 AM
Hi MTCAT
That size is definitely wrong. I forgot to mention that when you change user to root, it also executes:
Code: [Select]
cd /rootso you are now in user roots home directory.

After you become root, do this:
Code: [Select]
cd /home/tc/tempdirThen try the  find . | cpio ....  command again.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 23, 2021, 08:21:55 AM
Hi Rich,

Thanks !, I should have noticed that, changing user to root, and then cd to /home/tc/tempdir, executing the big "find . |   " command definitely gave the DX3 a workout ! Now tinycoreRT1.gz is 8209911 bytes, hopefully that should be good now !

Before I reboot though, is there a way to add "no restore" option in extlinux.conf ? Or maybe I should just remove "mydata.tgz" and store it somewhere else, right now it's loading the backup every boot and it takes quite a long time, and makes me uneasy or wondering if I'm overwriting something I don't want too with the backup.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 23, 2021, 08:34:42 AM
Hi MTCAT
... Before I reboot though, is there a way to add "no restore" option in extlinux.conf ? Or maybe I should just remove "mydata.tgz" and store it somewhere else, right now it's loading the backup every boot and it takes quite a long time, and makes me uneasy or wondering if I'm overwriting something I don't want too with the backup.
First copy your  mydata.tgz  to  mydata.tgz.RTkernel.  This way you have a backup should you ever need it.

Then do this:
Code: [Select]
sudo touch /tmp/dummy
sudo echo "tmp/dummy" > /opt/.filetool.lst

This will backup/restore an empty file which will be very quick.
This will allow you to still add files to be backed up if needed.

We covered this before :):
http://forum.tinycorelinux.net/index.php/topic,24813.msg157840.html#msg157840
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 23, 2021, 08:52:58 AM
Hi Rich,

Oh, right, sorry about that, I do recall now that "sudo touch" command but I didn't realize that's what we were doing (making an empty backup file). Thanks again, I will do that before I (hopefully) successfully boot up the RT kernel, which will be the next boot attempt !

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 23, 2021, 09:04:57 AM
Hi MTCAT
Oh, right, sorry about that, ...
Not a problem.

Quote
... I do recall now that "sudo touch" command but I didn't realize that's what we were doing (making an empty backup file). ...
That's on me. I didn't mention it in that post, but will add that information to it.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 23, 2021, 10:00:25 AM
Hi Rich,

No problem, good to know what's going on there is all.

So I tried to boot up the real time kernel and now get a different modprobe error;

modprobe: can't open 'modules.dep': No such file or directory
Failed to open /dev/ramzswap0: No such file or directory
swapon: can't stat '/dev/ramzswap0': No such file or directory

And then when it gets to loading extensions, it hangs up there again. I thought we dealt with modules.dep and modules.alias already ??

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 23, 2021, 10:05:21 AM
Hi MTCAT
Does it boot to a prompt? You may have to hit the carriage return a couple of times to see it.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 23, 2021, 10:27:57 AM
Hi Rich,

I do see a "boot:" prompt initially but I don't seem to have a choice as to what to boot, there's no "LILO like" list to choose from or anything ?

But I just wait a few moments and then it proceeds to boot the kernel indicated as default in extlinux.conf.

Thanks,

David

Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 23, 2021, 10:37:31 AM
Hi MTCAT
Sorry, I meant command prompt.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 23, 2021, 10:41:31 AM
Hi Rich,

No, unfortunately it's getting stuck again at the "Loading Extensions" step, the boot process never gets past that when trying to boot the real time kernel, I did copy all the *-rt19-tinycore.tgz files into /mnt/sdb1/tce/optional though, unless some are still missing somehow ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 23, 2021, 10:56:56 AM
Hi MTCAT
... But I just wait a few moments and then it proceeds to boot the kernel indicated as default in extlinux.conf.
So DEFAULT points to realtime, right?
And realtime looks like this:
Code: [Select]
LABEL realtime
KERNEL /boot/vmlinuzRT1
APPEND initrd=/boot/tinycoreRT1.gz quiet home=UUID= .....

If that all looks correct, add the word  text  like this:
Code: [Select]
APPEND initrd=/boot/tinycoreRT1.gz quiet text home=UUID= ..... This will skip loading extensions. See if you get to a command prompt then.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 23, 2021, 12:28:09 PM
Hi Rich,

Sorry for all these problems, the extlinux.conf is setup properly for realtime, I added the "text" entry to the APPEND line as you indicated but the boot process still tried to load extensions, and get's stuck there. I attached the extlinux.conf file here.

Thanks,

David

Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 23, 2021, 12:41:57 PM
Hi MTCAT
Try  base  instead of  text.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 23, 2021, 12:52:37 PM
Hi MTCAT
One more thing, on both entries, change this:
Code: [Select]
waitusb=5 waitusb=5:UUID="622fa3dd-62bb-43fc-a837-43021b9b3c7f"to this:
Code: [Select]
waitusb=10:UUID="622fa3dd-62bb-43fc-a837-43021b9b3c7f"
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 23, 2021, 01:02:43 PM
Hi Rich,

Using "base" worked !, I was able to boot into X now, and "uname -r" returns "2.6.33.3-rt19-tinycore", so we skipped loading extensions with the "base" modification ?

What now, add one extension at a time to see when booting fails maybe ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 23, 2021, 01:04:09 PM
Hi Rich,

Forgot to add, I'll change the waitusb to 10 as you instructed as well, I also forgot to do the sudo touch modification, will do both of those now.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 23, 2021, 01:13:26 PM
Hi MTCAT
Using "base" worked !, I was able to boot into X now, and "uname -r" returns "2.6.33.3-rt19-tinycore", so we skipped loading extensions with the "base" modification ? ...
Actually, what you booted into is called the console. X is a graphical environment.
Yes, the  base  boot code instructs Tinycore not to load any extensions.
The  text  boot code instructs Tinycore not to start  X  after loading extensions.

Quote
... What now, add one extension at a time to see when booting fails maybe ?
Attach a copy of your  onboot.lst  file and we will proceed from there.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 23, 2021, 01:46:59 PM
Hi Rich,

Thanks for all the help, I had problems mounting a second pen drive, so here is onboot.lst typed out for you !

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 23, 2021, 02:03:37 PM
Hi MTCAT
So you are not running a GUI. Lets start with a simple one:
Code: [Select]
tce-load -i zsync
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 23, 2021, 02:25:28 PM
Hi Rich,

The drama continues, for the RT kernel, all the extensions were downloaded previously with the non-RT kernel, and then the tgz files copied to /mnt/sdb1/tce/optional.

Since then, I've actually never had the ethernet up and going with the real time kernel.

With the rt kernel, using the "base" boot option, after booting into the GUI window manager, when I type "ifconfig -a" in a terminal window, I don't even see an "eth0" at all, I only see a "dummy0" and an "l0".

Is this because we're skipping something getting loaded for eth0 with the "base" boot option ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 23, 2021, 02:39:28 PM
Hi MTCAT
OK, so you can load extensions. Reboot to base again. When you get to the command prompt:
Code: [Select]
sudo depmod -a
sudo udevadm trigger
Then load your extensions. See if  eth0  shows up now,
Got to go. Back in about 2 hours.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 23, 2021, 03:59:29 PM
Hi Rich,

You're amazing, how do you remember all these commands, that worked !, after "sudo depmod -a" and "sudo udevadm trigger" eth0 appeared out of the ether (ha-ha) and then I applied the IP address's with the Network App !

My first try at using "tce-load -i zsync" didn't work though, I received an error "zsync.tgz not found !", but, I was able to install zsync with the Control Panel App, and then for the fun of it, I tried the command line in a terminal again, "tce-load -i zsync", and then I received back "zsync is already installed !"

So, hmm, not sure why the terminal prompt command didn't work the first time, but at least the Control App did work !

Maybe try loading another extension with "tce-load" ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 23, 2021, 09:33:45 PM
Hi MTCAT
... My first try at using "tce-load -i zsync" didn't work though, I received an error "zsync.tgz not found !", but, I was able to install zsync with the Control Panel App, and then for the fun of it, I tried the command line in a terminal again, "tce-load -i zsync", and then I received back "zsync is already installed !" ...
Extension names end in  .tcz  not  .tgz.
Are you sure  zsync.tcz  was in your  tce/optional  directory?

... On boot-up, I wasn't given a choice between the old kernel and the real-time kernel, I just saw a little "boot: " prompt echoed to screen, and then eventually, off it went with the real-time kernel. ...
It's possible the  TIMEOUT  setting in your  extlinux.conf  file is too short for you to make a selection. Change it to:
Code: [Select]
TIMEOUT 500That will give you 50 seconds instead of only 5. There should be some way to select what to boot, Maybe the
up/down arrows?

... modprobe: can't open 'modules.dep': No such file or directory ...
Let's try to address that. Boot into the RT Kernel. When you get the command prompt:
Follow the instructions in the attached file. The new initrd will be called  tinycoreRT2.gz  so we don't overwrite the
existing one. Adjust your extlinux.conf accordingly.

Reboot and see if eth0 is found now.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 24, 2021, 02:49:04 PM
Hi Rich,

Sorry I've been away from my computer all day, an annoying home improvement project, I'll boot the RT kernel, change the TIMEOUT and take a try at your new instructions to address modules.dep issue.

Sorry for my confusion re: the module extension, I just thought it was odd that the "manual" tce-load method (typed from within a terminal window) didn't initially work to install zsync, but the Control App method did work.

Also, just to make sure, with the RT kernel, I'm working from within the window manager, opening terminal windows and executing commands that way, is it fair to say that I'm technically not at a command prompt ? I wasn't sure if you thought that I wasn't getting into the window manager, and just booting to a command prompt only. I'm actually getting into the Windows manager (X ?) and opening terminal windows inside that.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 24, 2021, 03:50:49 PM
Hi Rich,

I made the new initrd "tinycoreRT2.gz", and increased TIMEOUT to 500, but, not sure, was I supposed to remove the "base" modification from the APPEND line as well ?

With the "base" still in place, the RT kernel boots up into the window manager and unfortunately, I don't see eth0, and I didn't see it on the first boot up today as well (using tinycoreRT.gz), even though you helped me get eth0 going last night, and I saved the changes in the Network App manager, it still didn't show up.

I confirmed that there is a call to "eth0.sh" in /mnt/sdb1/opt/bootlocal.sh, and eth0.sh is also present in /mnt/sdb1/opt, with the correct IP address's, etc., so that seems weird.

The increased TIMEOUT worked but unfortunately, the arrow keys didn't do anything at the "boot:" prompt, so I just hit enter and off it went with the RT kernel using tinycoreRT2.gz

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 26, 2021, 05:06:09 PM
Hi Rich,

I removed the "base" from the APPEND line and was able to boot w/o the "modules.dep" problem, but I still see the "ramzswap0" error. Also, I saw a strange error, after loading extensions, I saw this on the screen; "ifconfig: SIOCSIFADDR: No such device and "route: SIOCADDRT: No such process"

Eventually it still booted up but with no eth0, but when I did your "sudo depmod -a" followed by "sudo udevadm trigger", eth0 was there.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 26, 2021, 05:22:18 PM
Hi Rich,

Hopefully this isn't "too much information", but I attached dmesg output here as well.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 26, 2021, 05:51:15 PM
Hi MTCAT
I don't know what the problem is. Try running the  TCscan.sh  script after the GUI is up so I can see a little more about
the state of this system. If you forgot how, you did it here previously:
http://forum.tinycorelinux.net/index.php/topic,24813.msg157696.html#msg157696
Attach the results to your next post.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 27, 2021, 07:24:09 AM
Hi Rich,

Good idea, here's the ".tmp" file, had to attach it because the scan never finished, it couldn't find the bootloader config file and it just got stuck there.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 27, 2021, 11:36:14 AM
Hi MTCAT
Since you insist on breaking my script, I've added a  "Press enter to skip."  option when it gets to the  Find bootloader part.
I've attached it here since the server won't let me update it in the original thread.

Please run it twice.
Once when you get to the command prompt.
Then:
Code: [Select]
sudo depmod -a
sudo udevadm trigger
Run the  /opt/eth0.sh  script and start the GUI. Now run  TCscan  again.

Attach both files to your next post. I'll know which is which because the file names contain time stamps.
Also attach a copy of your  /opt/eth0.sh  file.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 27, 2021, 01:47:01 PM
Hi Rich,

Thanks for the new script, here's the outputs, before and after getting eth0 up and running. Just to be clear, I'm doing this all within a terminal window in FLWM though, hope that's okay.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 27, 2021, 01:53:43 PM
Hi Rich,

Sorry, forgot to attach eth0.sh, here it is, it looks a bit different than the non-RT kernel eth0.sh I think, I don't recall that "pkill" command for example.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 27, 2021, 06:28:34 PM
Hi MTCAT
The only thing I see is you don't appear to have a  mydata.tgz  file. I'm wondering if maybe that is causing an error
in the  tc-config  script and messing up something else.

Let's do this again:
Code: [Select]
sudo touch /tmp/dummy
sudo echo "tmp/dummy" > /opt/.filetool.lst

Now let's run a backup:
Code: [Select]
filetool.sh -bSee if it boots up correctly now.

... Just to be clear, I'm doing this all within a terminal window in FLWM though, hope that's okay.
I don't see  flwm_topside  or anything else required for a GUI in your  onboot.lst  file.

How are you starting your GUI?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 28, 2021, 04:18:41 AM
Hi Rich,

I followed your instructions below but unfortunately that didn't change the boot-up behavior, other than now loading a backup file, I still see the "raqmzswap0" error and the odd "ifconfig" error as well with no eth0 present unless I do the "sudo depmod-a" and "sudo udevadm trigger" workaround.

I'm not doing anything special to start up the Window manager, that's just what gets booted up for me somehow, this is the same thing with my non-RT version, it just always boots into FLWM.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Juanito on April 28, 2021, 05:37:19 AM
At the risk of repeating something that has already been mentioned..

Once you have your new initrd ready to go with the rt kernel modules that go in the base and assuming the initrd is extracted to /tmp/extract, then you need this command before creating your new initrd:
Code: [Select]
$ sudo depmod -a -b /tmp/extract 3.6.4-tinycore
Where you substitute your kernel name for "3.6.4-tinycore" - after issuing the command you can delete the files created except for modules.alias and modules.dep and then create your new initrd.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 28, 2021, 05:48:13 AM
Hi Juanito
Good thought, but I already made him do that twice.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 28, 2021, 05:55:55 AM
Hi MTCAT
... I'm not doing anything special to start up the Window manager, that's just what gets booted up for me somehow, ...
There is something very wrong here. The contents of your  onboot.lst  file do not support that statement. Something else
is interfering with the boot process.

Unplug the data cable to the hard drive. Unplug the other thumb drive (sdc1). Boot up the system.
I expect it to boot up to a black screen with a command prompt.
See if  eth0  is present.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 28, 2021, 06:06:58 AM
Hi Rich,

I have only a CF-card for my "hard-drive", I'll remove it on next boot-up, and I have just the boot pen drive plugged in otherwise...

Not sure it helps or not, but I did find this https://www.kabisa.nl/tech/xen-how-to-fix-siocsifaddr-no-such-device/ regarding the eth0 issue, MAC address problem ?

Also, I found this re: the boot-loader https://www.linuxsecrets.com/tinycorelinux-wiki/wiki:extlinux.html , need to install vesamenu.c32 and chain.c32 in order to select the kernel at boot time ?

Sorry, off to work, will try the boot-up w/o the CF-card tonight.

Thanks for all the help Rich and Juanito,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 28, 2021, 06:34:29 AM
Hi MTCAT
... Not sure it helps or not, but I did find this https://www.kabisa.nl/tech/xen-how-to-fix-siocsifaddr-no-such-device/ regarding the eth0 issue, MAC address problem ? ...
No, I don't think so. I think it's related to the  ramzswap0  error, both of which are probably due to kernel modules not
being found. I suspect that is also related to your GUI magically coming up without you having to do anything.

Quote
... Also, I found this re: the boot-loader https://www.linuxsecrets.com/tinycorelinux-wiki/wiki:extlinux.html , need to install vesamenu.c32 and chain.c32 in order to select the kernel at boot time ? ...
Seems the writeup I looked at left out that little detail. Copy  vesamenu.c32  to the same directory as your  extlinux.conf
file. Add this to the top of your  extlinux.conf  file:
Code: [Select]
UI vesamenu.c32
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 28, 2021, 07:34:14 AM
Hi MTCAT
Before you change anything. After you've done a  depmod  and  udevadm, please confirm that the system can
now find the  zram  kernel module:
Code: [Select]
modinfo -d zramIf it returns a description, it found it. If it just returns a command prompt, it didn't.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 28, 2021, 05:52:35 PM
Hi Rich,

I removed the CF-card and unfortunately still saw the same behavior. I did the depmod and udevadm commands followed by the "modinfo -d zram" which returned no description at all, nothing was found, just returned to the command prompt.

I was poking around on the net and found this http://distro.ibiblio.org/tinycorelinux/3.x/contrib/rt-kernel/

Might there be anything in there that might help us ?, I notice that whoever built the kernel (Arslan ?), used the same 2.6.33.3-rt19 patch that I'm trying to use as well !

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 28, 2021, 05:57:58 PM
Hi Rich,

I don't know if this helps or not, but I attached the output of lsmod too, it looks kind of weird to me, very different anyway than the non-RT kernel, but maybe there's a hint there for someone who knows what to look for !

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 28, 2021, 06:47:57 PM
Hi MTCAT
According to  the last scan you ran, there are 3 devices connected, sda, sdb, and sdc. Did you disconnect both
devices that do not contain the real time kernel?

With both devices removed, is it still booting straight to a GUI?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: curaga on April 28, 2021, 11:57:26 PM
Oh indeed, there's a ready RT kernel with your version, and the extensions already in the 3.x repo. Do try that instead of making your own.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 29, 2021, 09:28:09 AM
Hi Rich and curaga,

Yup, I disconnected the CF-card (usually labeled sda5) and then just booted up with the pen drive containing the RT kernel, the boot process gets hung up on the ifconfig error for a while, but then kicks out of that somehow and goes right into FLWM.

I tried to install vesamenu.c32 but I downloaded the wrong version it seems, I think I can get the right version from the "syslinux" package, so I'll try that next, just for the heck of it.

What do you think about using a pre-built RT kernel though ? Would that be as simple a process as installing "regular" TinyCore, like we did when working from an iso file ?, the pre-built kernel is just the "bzImage" file and the modules (I think).

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 29, 2021, 10:57:36 AM
Hi MTCAT
Yup, I disconnected the CF-card (usually labeled sda5) and then just booted up with the pen drive containing the RT kernel, the boot process gets hung up on the ifconfig error for a while, but then kicks out of that somehow and goes right into FLWM. ...
sda  is probably this:
Code: [Select]
/proc/swaps=
Filename Type Size Used Priority
/dev/sda1                               partition 1043448 0 -1

sdb  is your thumb drive with the RT kernel. What is  sdc:
Code: [Select]
mounts=
rootfs on / type rootfs (rw,relatime,size=1803816k,nr_inodes=218376)
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,relatime)
devpts on /dev/pts type devpts (rw,relatime,mode=600)
tmpfs on /dev/shm type tmpfs (rw,relatime)
/dev/sdb1 on /mnt/sdb1 type ext4 (rw,relatime,barrier=1,data=ordered)
/dev/sdb1 on /home type ext4 (rw,relatime,barrier=1,data=ordered)
/dev/sdb1 on /opt type ext4 (rw,relatime,barrier=1,data=ordered)
/dev/sdc1 on /mnt/sdc1 type vfat (rw,relatime,fmask=0000,dmask=0000,allow_utime=0022,codepage=cp437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)

disk free=
Filesystem                Size      Used Available Use% Mounted on
rootfs                    1.7G     20.1M      1.7G   1% /
tmpfs                   978.6M         0    978.6M   0% /dev/shm
/dev/sdb1                14.1G      1.4G     12.7G  10% /mnt/sdb1
/dev/sdb1                14.1G      1.4G     12.7G  10% /home
/dev/sdb1                14.1G      1.4G     12.7G  10% /opt
/dev/sdc1                59.7G     57.0G      2.7G  96% /mnt/sdc1
From the  dmesg  section:
Code: [Select]
scsi3 : usb-storage 1-2:1.0
scsi 3:0:0:0: Direct-Access     SanDisk  Cruzer Glide     1100 PQ: 0 ANSI: 6
sd 3:0:0:0: Attached scsi generic sg2 type 0
sdc: sdc1
sd 3:0:0:0: [sdc] Attached SCSI removable disk

Quote
... I tried to install vesamenu.c32 but I downloaded the wrong version it seems, ...
If you used the  tc-install  extension from TC3, you need to install the  syslinux.tcz  from TC3. Then copy:
Code: [Select]
/usr/local/share/syslinux/vesamenu.c32
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 29, 2021, 11:17:13 AM
Hi Rich,

sdc was a 64 GByte SanDisk pen drive that I use to transfer files between my main desktop and the little VortexDX3, but I didn't stick that in until after boot-up.

Thanks for the help re: syslinux, I clued into that as well this morning and downloaded syslinux from the repository (after getting eht0 up and running), copied vesamenu.c32 to /mnt/sdb1/boot/extlinux and it worked !, looks pretty nice too.

What do you think, try the pre-built RT kernel ?, but how to install with only the bzImage and module file (see link in previous post), or keep trying with the existing RT kernel ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 29, 2021, 02:18:51 PM
Hi MTCAT
... What do you think, try the pre-built RT kernel ?, but how to install with only the bzImage and module file (see link in previous post), or keep trying with the existing RT kernel ?

You'll need to include the modules file with the initrd somehow. I don't know if extlinux accepts multiple initrds, but I found
this post for combining the modules file with the stock initrd:
Grub is ok with merely cat'ed initrds though. cat tinycore.gz rt-modules.cpio.gz > tinycorert.gz

Less space efficient than repacking, but easier to do.

I don't know if you'll have any issues if you need to use any kernel module extensions.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 29, 2021, 06:16:24 PM
Hi Rich,

Starting from scratch, would the general idea be, boot up TC3 (on my old YUMI pen drive), install regular TC3.8.4 from the iso onto a new pen drive as per your instructions, then place the real-time bzImage in /mnt/sdb1/boot on the freshly made TC3.8.4 install, make the new initrd as you just showed me (with cat, using the existing initrd and the rt cpio,gz file), lastly modify extlinux so as to load the RT kernel and use the RT initrd and optionally add vesamenu.c32 so as to be able to choose between RT kernel and regular kernel at boot up.

Does that sound right ?

Thanks,

David



 
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on April 29, 2021, 06:27:33 PM
Hi MTCAT
Sounds right.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on April 29, 2021, 07:24:04 PM
Hi Rich,

Mark it on the calendar, I got something right :), I'll try that tomorrow hopefully, or Saturday at the latest ! Will see what happens....

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 01, 2021, 09:31:32 AM
Hi Rich and curaga,

Installing the pre-built RT kernel and modules worked !, no problems !, everything went perfectly, "uname -r" returns "2.6.33.3-l1-rt19". I've got home and opt set up as persistent, ethernet is going, the bootloader is working properly, all is well !

Now I can work through getting the A-D driver installed again, chrony for GPS time sync, and last step will be to get the actual data acquisition program going !, getting close to finishing the recipe, amazing....

I suppose if I really want to be a "keener", I could try testing the RT kernel somehow re: latencies or something, but I'm too happy with myself at the moment to think too hard about that !

Thanks for all the help.

Cheers,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 01, 2021, 10:28:20 AM
Hi Rich,

For something easy to do, I thought I would install all the extensions (pci-utils, compiletc, etc.), I don't know if it's an issue with my internet, or the tinycore server, but it was taking forever to install compiletc, and it didn't finish properly, base-dev.tcz timed out (md5 sum error or something), after exiting the install (App Finder), I instead typed at a prompt "tce-load -wi compiletc" and got a message, "extension  already installed", yet I know it didn't finish properly.

Is there a way I can uninstall the (partially installed) compiletc package so that I can re-do it properly ?

Typing gcc -v the C compiler does seem to be there, but I don't want to take any chances.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 01, 2021, 11:10:26 AM
Hi MTCAT
First, remove  compiletc.tcz  from your  onboot.lst  file and reboot.

Remove the files listed in the  .tree  file from your  tce/optional  directory:
http://repo.tinycorelinux.net/3.x/tcz/compiletc.tcz.tree
Some of the names are listed more than once. That's normal for a  .tree  file. Don't forget to also remove the matching
.tcz.md5.txt  and  .tcz.dep  files.

Then, one at a time, install (tce-load -wi) the files listed in the  .dep  file:
http://repo.tinycorelinux.net/3.x/tcz/compiletc.tcz.dep
Then see where it chokes.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 01, 2021, 01:31:50 PM
Hi Rich,

Thanks for the help, I followed your instructions to clean out all the .tcz, .md5 and .dep files, and then started re-loading them one by one. I downloaded binutils.tcz but then I got hung up on base-dev.tcz again, it downloaded 55 % of it and then "stalled".... and then just sits there at that.

I suppose I could copy over all the extensions from my other install of non-rt tinycore, but that seems like I shouldn't have to do that.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 01, 2021, 02:19:36 PM
Hi Rich,

Something weird here, when I type "tce-load -iw perl5" for example, I may get somewhere between 30 to 50 percent done, then it really slows down, and eventually stalls.

But, if I open another terminal, and type "ping 8.8.8.8" before the download stalls, it seems to "wake-up" the download and then it continues on ????

Smaller packages seem to download okay without needing the extra "ping" command, it's almost like I can only download so much data before it stalls, around 3 MBytes.

I was able to fully download base-dev.tcz this way, with the extra "ping" command, and all the others needed for compiletc too, but not sure why I should have to "wake-up" my eth0 with a ping command to force it to keep going (not to stall) ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 01, 2021, 02:27:29 PM
Hi MTCAT
See if this helps:
http://forum.tinycorelinux.net/index.php/topic,24804.msg157651.html#msg157651
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 01, 2021, 03:00:34 PM
Hi Rich,

Thanks for that extra command, I had ping going continuously in another window and already finished downloading all my extensions that way, but I tried your command and then downloaded ntp for the fun of it and it seemed to work, although ntp is actually not that big.

Can you think of a reason why we would need to do this with the RT kernel ? The "tce-load" always worked really well with my previous non-RT installs.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 01, 2021, 07:06:59 PM
Hi MTCAT
A standard kernel allows a process to run for a period of time, then stops it to allow the next process to run, then stops
that process to allow yet another process to run, and so on. This works well for a desktop setup. It gives every process
run time and is well suited for having multiple applications running at the same time. Applications politely share run time.

A preemptive real time kernel is designed to allow a process to run with minimal delay. Even if it means preempting
another process. This is good for a time critical dedicated process, but not for a desktop system that relies on
cooperation and sharing.

My guess is the download process is hogging all the time and possibly interfering with some housekeeping the system
needs to perform. That's just a guess on my part.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 02, 2021, 09:43:07 AM
Hi Rich,

Thanks, maybe that's what's going on....possible downside to the RT kernel I guess....

Would "tce-audit builddb" followed by "tce-audit delete compiletc" work to remove compiletc as well ?, even if compiletc was only partially installed, found a thread on that yesterday.

I guess if the 100 MBps ethernet port becomes a bit of a pain (for offloading data from the instrument), we could always just use the USB 2.0 ports on the VDX3, assuming the RT kernel doesn't decide to also preempt the USB copying ! The VDX3 also has a 1000 MBps ethernet port, but, TC 3.8.4 doesn't seem to see it.

I should be able to apply the same process I did yesterday to my non-RT kernel (with 24dsi.ko and chrony going), to "upgrade" that install to RT as well ? Although, I should probably go through all the steps to build the AD driver and install etc, and autoload at boot with the RT pen-drive, just for the practice, and so that I can clean up all my notes.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 02, 2021, 10:17:43 AM
Hi MTCAT
... Would "tce-audit builddb" followed by "tce-audit delete compiletc" work to remove compiletc as well ?, even if compiletc was only partially installed, found a thread on that yesterday. ...
It might. I don't know the exact sequence of commands for that. If you use the  Apps  utility you should be able to do it by
clicking  Apps->Maintenance->Dependencies And Deletions. Then select  compiletc.tcz in the left panel. Then click
Dependencies->Mark for Deletion. Then restart the system and they should be gone.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 04, 2021, 06:25:30 PM
Thanks Rich, I'm hoping to try building the 24DSI driver and making the out of tree package, loading it up at boot time, etc., on Thursday, it's been driving me nuts to have to wait that long though.....
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 06, 2021, 01:22:31 PM
Hi Rich,

Embarrasingly, I'm having problems already with building the 24DSI driver on the RT kernel, see attached text file for what I did so far, I must be forgetting something.

After running "./make_all", or "sudo ./make_all", I'm seeing

driver==================================================
make: ***/lib/modules/2.6.33.3-l1-rt19/build: No such file or directory. Stop.
make: ***[24dsi.ko] Error 2
Driver loading: 24dsi ....
--->ERROR<----: module file does not exist: /home/tc/24dsi.linux.4.11.91.32.0/24dsi/./driver/24dsi.ko

I do have linux-headers.2.6.33.3-tinycore.tcz installed, sorry about this, I must be forgetting something.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 06, 2021, 05:18:51 PM
Hi MTCAT
... I do have linux-headers.2.6.33.3-tinycore.tcz installed, sorry about this, I must be forgetting something.

Take a look down this path and see what's wrong/missing:
Quote
make: ***/lib/modules/2.6.33.3-l1-rt19/build: No such file or directory. Stop.

You may want to review the thread where we hashed all this out. As I recall, it's about 10 pages long, and I don't
remember all of the details.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 06, 2021, 07:09:35 PM
Hi Rich,

Thanks for the help, I printed out the whole thread some time ago for reference and I went through it all but I'm not sure yet what's missing. I'm concerned that my RT installation is perhaps not 100 percent correct, I hope that's not the case though.

On the original non RT install, when I "cd lib/modules/2.6.33.3-tinycore" and type "ls -l" I see (amongst others);

lrwxrwrwx 1 root root 79 May 7 01:29 build -> tmp/tcloop/linux-headers-2.6.33.3-tinycore/lib/modules/2.6.33.3-tinycore/build

And when I "cd build" and then type "pwd"I get

"/lib/modules/2.6.33.3-tinycore/build" but yet the prompt shows "tc@box:/usr/local/src/linux-headers-2.6.33.3-tinycore$"

But on the RT pen-drive, when I "cd lib/modules/2.6.33.3-l1-rt19" there's no build directory in there at all. But, still on the RT pen drive, when I "cd lib/modules/2.6.33.3-tinycore" (leftover from the nonRT kernel that I installed first before adding in the RT bzImage and making a new initrd), there is a build directory in there !

The first time around I was fixated on installing the driver to /usr/src/linux/drivers as per the GSC instructions, so after we installed linux-headers-2.6.33.3-tinycore.tcz, I also made this directory "mkdir /usr/src/linux/drivers", and worked out of there originally to build the driver,  but later on switched over (as per your guidance) to building the driver in home directory, to keep the driver build persistent, so that's what I tried to do this time around, just started right off the bat by building the driver in /home/tc/24.dsi.linux.4.11.91.32.0...that's about all I can see that is different this time around, besides the missing "build" directory in the RT modules directory.

I did verify that Module.Symvers is in /usr/src/linux on the RT pen drive but for some reason I'm missing this "build" directory under "/lib/modules/2.6.33.3-l1-rt19", any idea why it would be missing in the RT lib/modules directory ?

Also, on the RT pen drive, when I "cd /usr/src/linux", I once again see -----> "tc@box:/usr/local/src/linux-headers-2.6.33.3-tinycore$" at the terminal prompt, is that right ?, or shouldn't it be "..../linux-headers-2.6.33.3-l1-rt19" if the RT was installed properly ?

Sorry to be such a pain.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 08, 2021, 05:43:20 AM
Hi MTCAT
... But on the RT pen-drive, when I "cd lib/modules/2.6.33.3-l1-rt19" there's no build directory in there at all. ...
That's because you don't have  linux-headers.2.6.33.3-l1-rt19.tcz.  It doesn't exist. There's probably a script for
creating it. Maybe curaga or Juanito can shed some light on this.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Juanito on May 08, 2021, 06:47:58 AM
Maybe I’m wrong, but I would have thought the headers are the same?

The build directory is just a symlink to the configured kernel source.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 08, 2021, 08:31:51 AM
Hi Rich and Juanito,

Thanks so much for the help; I was wondering what you guys think about the following,

1) I assume the 24DSI driver we already built for 2.6.33.3 will not work with 2.6.33.3-l1-rt19 due to kernel mismatch ?

2) Is there a way I can make a new symbolic link with "sudo ln -s ????" inside  /lib/modules/2.6.33.3-l1-rt19 ? Kind of like
this person did here http://forum.tinycorelinux.net/index.php/topic,23847.0.html

3) Is there a way to make a new package "linux-headers.2.6.33.3-l1-rt19.tcz" that could be installed ?

4) In an older post from Juanito here http://forum.tinycorelinux.net/index.php/topic,7219.0.html and also here
http://forum.tinycorelinux.net/index.php/topic,23847.0.html, could I  "expand" the bzImagert kernel file and
then make a symbolic link for the RT kernel that way ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Juanito on May 08, 2021, 08:46:29 AM
Try to load your driver - you will get an error message if the kernel doesn’t like it.

To compile a new driver, I believe you can use the standard kernel headers, but you will need to prepare the kernel source with the real time kernel config.

Yes, you can make the build symlink as described.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 08, 2021, 11:14:20 AM
Hi Rich and Juanito,

I brought the RT kernel to my old non-RT pen drive, modified extlinux.conf appropriately, and tried to load the 24DSI driver with the RT kernel, unfortunately this didn't work, I saw --->ERROR<--- flash past during the driver load attempt, and sure enough, "lsmod" doesn't show the 24dsi module there.

So it looks like I need to build the driver with the RT kernel somehow.

I apologize for my lack of knowledge/experience but could you advise as to what would be the best way to get the "build" symbolic link into /lib/modules/2.6.33.3-l1-rt19" directory ?

Might it be as easy as one "sudo ln-s xxxxx" command ? Or do I need to unpack the bzImagert file and then make a symbolic link ??

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Juanito on May 08, 2021, 11:35:16 AM
It’s as easy as “sudo ln -s xxx”.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 08, 2021, 12:20:12 PM
Hi Juanito,

Thanks for the help, what do I point the link too though ???, i.e., can you help me fill in the "xxxxx" below ?

sudo ln -s xxxxxx /lib/modules/2.6.33.3-l1-rt19/build

I don't have the kernel source to "point" too, I only have the pre-built bzImagert file.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Juanito on May 08, 2021, 12:50:47 PM
You can’t use bzImage, you need the kernel source.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 08, 2021, 02:07:32 PM
Hi Juanito,

Is there any way of accomplishing the end goal, namely, getting a symbolic link /lib/modules/2.6.33.3-l1-rt19/build without the kernel source ?

When you say I need the kernel source, I think you are saying that I need to build the kernel from scratch ?

I already tried that and wasn't completely successful, I could try again I suppose, but I was hoping to use the pre-built kernel, the bzImagert file already made for us.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 08, 2021, 05:13:51 PM
Hi Rich and Juanito,

I thought I would try to install the 24DSI driver in the "do-it-yourself" RT kernel that Rich helped me build from scratch
a few days ago.

This worked ! With the DIY RT kernel, I was able to build the 24DSI driver, load the module and run some of the sample programs
to test the A-D ! Finally some good news.

So now the question is, should we go back to trying to fix up the DIY RT kernel install (still some problems --> "modinfo -d
zram" returns nothing, there's no eth0 unless we run "sudo depmod -a" followed by "udevadm trigger", and FLWM seemingly
starts on it's own), or, try to modify the pre-made RT kernel by somehow adding the symbolic link
"/lib/modules/2.6.33.3-l1-rt19/build" so that I can install the driver w/o error.

What do you guys think ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 08, 2021, 07:17:06 PM
Hi MTCAT
... So now the question is, should we go back to trying to fix up the DIY RT kernel install ...
No, move the kernel module to the working RT install. The kernel module should now match the RT kernel so you
shouldn't need the build directory anymore.

Quote
... and FLWM seemingly starts on it's own) ...
I figured out why that was happening. I unpacked the initrd for 3.8.4. Turns out, back then flwm_topside was included
in the initrd.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Juanito on May 09, 2021, 05:18:32 AM
When you say I need the kernel source, I think you are saying that I need to build the kernel from scratch ?

No, you need the kernel source tarball and the rt kernel config so that you can prepare the kernel source for an out of tree module to build against it.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 09, 2021, 05:42:04 AM
Hi MTCAT
linux-2.6.33.3-patched.tbz2 tarball:
http://repo.tinycorelinux.net/3.x/release/src/kernel/

Kernel config:
http://repo.tinycorelinux.net/3.x/contrib/rt-kernel/src/
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 09, 2021, 02:17:44 PM
Hi Rich and Juanito,

Thanks again for all the help, Rich, for the DIY RT kernel on which I built the 24DSI driver last night, "uname-r" returns  "2.6.33.3-rt19-tinycore", but on the pre-built RT kernel, "uname-r" returns "2.6.33.3-l1-rt19", modprobe will be okay with that ?

Alternatively, if I needed to compile the 24DSI driver on the pre-built RT kernel (2.6.33.3-l1-rt19) I would need to unpack the "linux-2.6.33.3-patched.tbz2" tarball, in home/linux-2.6.33.3-patched say, and then type "sudo ln -s /home/linux-2.6.33.3-patched /lib/modules/2.6.33.3-l1-rt19/build", is that right ?

But there must be more to it though as I still need to make use of the pre-built rtkernel config file somehow ??

Speaking of which, for the DIY RT kernel build that I attempted, I wonder if I used the wrong config file ?, this is the one I used, "wget http://tinycorelinux.net/3.x/release/src/kernel/config-2.6.33.3-tinycore"

I also attached a text file of the steps I did in the DIY build, read on if you have the stomach to do so !, sorry, probably no end to the mistake  there, but may explain the odd behavor of the DIY RT kernel.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Juanito on May 10, 2021, 12:42:52 AM
I assume that you need to download linux-2.6.33.3-patched.tbz2, patch-2.6.33.3-rt19.bz2 and config-2.6.33.3-l1-rt19.

Uncompress linux-2.6.33.3-patched.tbz2 and patch-2.6.33.3-rt19.bz2 and apply the patch.

Load compiletc, perl5, bash and ncurses-dev

Then prepare the kernel:
Code: [Select]
$ cd linux-2.6.33.3
$ make mrproper
$ cp ../config-2.6.33.3-l1-rt19 .config
$ make oldconfig
$ make modules

If you can find Module.symvers there is a quicker way to do this.

Finally make the build symlink and then you should be able to compile your module
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 10, 2021, 05:43:03 PM
Hi Juanito,

Thanks for the extra detail, much appreciated. After the "make modules" command, do I need to set the module install path, or do anything else ?, or would it just be

make modules
sudo ln -s /home/linux-2.6.33.3-patched /lib/modules/2.6.33.3-l1-rt19/build"

Regarding Module.symvers, there is a file so named in /usr/src/linux-headers-2.6.33.3-tinycore on the pre-built RT kernel pen drive, this is from the original non RT install I think (see step 2 in list attached previously), is that the right file ?, if so, can you show me the quicker way to do it as well ?

I will also try Rich's suggestion re: loading the 24dsi.ko built on my DIY RT kernel, but it seems like it's always good to have multiple possible ways to solve a problem, just in case one (or more) ways don't work !

Thanks so much,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 10, 2021, 07:30:13 PM
Hi MTCAT
... Regarding Module.symvers, there is a file so named in /usr/src/linux-headers-2.6.33.3-tinycore on the pre-built RT kernel pen drive, this is from the original non RT install I think ...
I think you would want one created when your kernel was created. Maybe it was included in the initrd? Check here:
Code: [Select]
ls -l /lib/modules/2.6.33.3-l1-rt19
Quote
... I will also try Rich's suggestion re: loading the 24dsi.ko built on my DIY RT kernel ...
The version strings don't match so the kernel may reject it. Your kernel is  2.6.33.3-l1-rt19  but your module is something
like  2.6.33.3-rt19-tinycore.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 11, 2021, 06:49:25 PM
Hi Juanito and Rich,

I did as you instructed in reply 120 Juanito, it took about 1.5 hrs to run "make -j2 modules" !, after it was done, I typed "sudo ln -s /home/linux-2.6.33.3 /lib/modules/2.6.33.3-l1-rt19/build".

Now, when I "cd /lib/modules/2.6.33.3-l1-rt19/modules" and type "ls -l" I can see ,

lrwxrwxrwx  1  1  root root  20 May 12  01:37  build --->  /home/linux-2.6.33.3

So that's exciting, does that sound right ?

In /usr/src I see a "linux" directory, after typing "cd linux", I see /usr/local/src/linux-headers-2.6.33.3-tinycore as the directory, in which I see a "Modules.symvers" in there, does this "Modules.symvers" need to be replaced with the output of "make -j2 modules" ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 11, 2021, 07:13:21 PM
Hi guys,

Sorry for the typo in the last post,

"Now, when I "cd /lib/modules/2.6.33.3-l1-rt19/modules" and type "ls -l" I can see , "

Should be

"Now, when I "cd /lib/modules/2.6.33.3-l1-rt19" and type "ls -l" I can see"

Also, I noticed that my symbolic link didn't survive a power-down-power-up cycle, after the power-up, I had to re-do the "sudo ln -s ...." command to get the build directory back.

Thanks,

David





Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 11, 2021, 08:43:02 PM
Hi MTCAT
... Also, I noticed that my symbolic link didn't survive a power-down-power-up cycle, after the power-up, ...
All files and links to files in  /bin, /dev, /etc, /lib, /root, /sbin, /tmp, /usr, /var  are in RAM and get recreated every time you boot.
The contents for those directories come from the  initrd (core.gz)  and  .tcz  extensions.

Quote
... I had to re-do the "sudo ln -s ...." command to get the build directory back.
You can add that command to your  bootlocal.sh  file.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Juanito on May 12, 2021, 01:13:42 AM
In /usr/src I see a "linux" directory, after typing "cd linux", I see /usr/local/src/linux-headers-2.6.33.3-tinycore as the directory, in which I see a "Modules.symvers" in there, does this "Modules.symvers" need to be replaced with the output of "make -j2 modules" ?

No, the real time source has been prepared in /home/linux-2.6.33.3 and by creating the symlink:

/lib/modules/2.6.33.3-l1-rt19/modules/build --->  /home/linux-2.6.33.3

..your 24dsi.ko module will build against this.

You should copy /home/linux-2.6.33.3/Modules.symvers to a safe place, it will save you two hours if you need to prepare the kernel source again.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 12, 2021, 05:57:05 PM
Hi Juanito and Rich,

Great !, thanks a lot, I'll try building the 24DSI module tomorrow (after making the symbolic link of course), thanks for the suggestion to add it to bootlocal.sh. I'll copy Modules.symvers onto another pen drive for safekeeping too.

Thanks,

David

Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 13, 2021, 10:55:30 AM
Hi Juanito and Rich,

After making the symbolic link I tried to build the 24DSI driver and still get the same error, no build directory, even though it is there.

But, I do notice though, in /lib/modules/2.6.33.3-l1-rt19 I have

build --> /home/linux-2.6.33.3

But in /lib/modules/2.6.33.3 I have

build --> /tmp/tcloop/linux-headers-2.6.33.3-tinycore/lib/modules/2.6.33.3-tinycore/build

I think the "tcloop" refers to something that has been loaded into RAM ? My new symlink in /lib/modules/2.6.33.3-l1-rt19 is not "tclooped" so to speak, is this maybe why "sudo ./make_all" still can't find the build directory for 2.6.33.3-l1-rt19 ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Juanito on May 13, 2021, 11:25:36 AM
It looks like you booted the standard kernel rather than the real time kernel?

What does “uname -a” give?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 13, 2021, 12:47:03 PM
Hi Juanito,

I'm using vesamenu.c32 to select either the real-time kernel, or the regular kernel, I select the real-time kernel (which is also set to DEFAULT), once booted up, "uname -a"  returns "2.6.33.3-l1-rt19"

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 13, 2021, 12:59:41 PM
Hi Juanito,

Sorry, I gave you the output of "uname -r", I powered up again and booted up the real-time and typed "uname -a" as you requested, this is what I get;

Linux box 2.6.33.3-l1-rt19 #1 SMP PREEMPT RT Mon Mar 28 22:43:50 UTC 2011 i686 i686 i386 GNU/Linux

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 13, 2021, 02:34:50 PM
Hi MTCAT
After making the symbolic link I tried to build the 24DSI driver and still get the same error, no build directory, even though it is there. ...
Is that the entire error message (rhetorical)? Please provide exact error messages.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 13, 2021, 03:53:12 PM
Hi Rich,

Here's the error message;

driver=============================================================================
make: *** /lib/modules/2.6.33.3-l1-rt19/build: No such file or directory. Stop.
make: *** [24dsi.ko] Error 2
Driver loading: 24dsi...
--->ERROR<----: module file does not exist: /home/tc/24dsi.linux.4.11.91.32.0/24dsi/./driver/24dsi.ko

Also attached a screenshot, why is there a "dot" before "/driver/24dsi.ko" ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 13, 2021, 03:54:36 PM
Sorry, forgot to attach the screen-shot, here it is.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 13, 2021, 08:59:45 PM
Hi MTCAT
Do this:
Code: [Select]
cd /lib/modules/2.6.33.3-l1-rt19/build
ls -l > buildDir.txt
Attach the file to your next post.

... Also attached a screenshot, why is there a "dot" before "/driver/24dsi.ko" ?
One of the make files is prepending the  driver  directory name with  ./.  The  ./  syntax means look for what follows in
the current directory.

Create a script called  ls  like this:
Code: [Select]
echo "echo This is not the command you are looking for." > ls
chmod 775 ls

If you execute  ls  you'll get a directory listing as expected.
If you execute  ./ls  you've told the OS to look for  ls  in the current directory.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 14, 2021, 05:41:00 AM
Hi Rich,

Thanks for the help, I made the symbolic link as usual, "sudo ln -s /home/linux-2.6.33.3 /lib/modules/2.6.33.3-l1-rt19/build", but, when I try to "cd /lib/modules/2.6.33.3-l1-rt19/build" I can't do so, I get returned;

sh: cd: can't cd to /lib/modules/2.6.33.3-l1-rt19/build

I tried this both as regular user and root with the same result as above.

Inside /lib/modules/2.6.33-l1-rt19 the "build" link shows up as;

lrwxrwxrwx 1 root root  20 May 14 12:18 build -->/home/linux-2.6.33.3

So, not sure, the link is there but I can't use it for some reason ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 14, 2021, 05:44:26 AM
Hi MTCAT
What does this return:
Code: [Select]
ls -l /home/linux-2.6.33.3
Maybe that should be:
Code: [Select]
sudo ln -s /home/tc/linux-2.6.33.3 /lib/modules/2.6.33.3-l1-rt19/build
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 14, 2021, 05:55:08 AM
Hi Rich,

Thanks, you could be on to something, typing "ls -l /home/linux-2.6.33.3" I get

ls: cannot access /home/linux-2.6.33.3: No such file or directory

But, "ls -l /home/tc/linux-2.6.33.3", everything shows up, I'll try the new symbolic link including "tc" and see if that works !

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 14, 2021, 06:00:54 AM
Hi MTCAT
I've made this statement to others in the past, but I feel it bears repeating:
You are no longer in high school. Spelling counts. Punctuation counts. Capitalization counts. :)
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 14, 2021, 06:08:18 AM
Thank you so much Rich and Juanito, that worked !, some progress, I was using the wrong path in the symbolic link, I think (hope) it should be good now, I think I just need to change "insmod" command in the "start" script with "modprobe" and hopefully that will work 100 percent.

After doing the symbolic link with the right path, I can now build the 24dsi.ko module (awesome !) in the "pre-built" RT kernel but it unfortunately fails to load the module with "insmod", I'm seeing now an error later on in the build process when it actually tries to load the module;

Driver loading: 24dsi ...
insmod: can't insert 'home/tc/24dsi.4.11.91.32.0/24dsi/./driver/24dsi.ko': Invalid module format
--->ERROR<---: Unable to load module.

Darn-it, have to go to work now, maybe I should do the "auto-load" modifications to the start script, which includes replacing insmod with modprobe, and then see if that loads the module okay ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 14, 2021, 06:12:41 AM
Hi MTCAT
What does this return:
Code: [Select]
modinfo /home/tc/24dsi.4.11.91.32.0/24dsi/./driver/24dsi.ko
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 14, 2021, 06:13:30 AM
Hi Rich and Juanito, sorry, I was thinking of "~" as "/home" only when in fact it's "/home/tc", my apologies for the extra hassle.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 14, 2021, 06:19:21 AM
Hi Rich,

"modinfo /home/tc/24dsi.4.11.91.32.0/24dsi/./driver/24dsi.ko" just returns,

filename: /home/tc/24dsi.4.11.91.32.0/24dsi/./driver/24dsi.ko

Doesn't give any information about which kernel it was compiled for, I think modinfo is supposed to do that as well ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 14, 2021, 06:22:25 AM
Hi MTCAT
That suggests  /home/tc/24dsi.4.11.91.32.0/24dsi/./driver/24dsi.ko  does not exist.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 14, 2021, 06:35:32 AM
Hi Rich,

Sorry, I missed ".linux" in the path,

"modinfo 'home/tc/24dsi.linux.4.11.91.32.0/24dsi/driver/24dsi.ko" returns,

filename: /home/tc/24dsi.linux.4.11.91.32.0/24dsi/driver/24dsi.ko
description: 24DSI driver 4.11.91.32.0
author: General Standards Corporation, www.generalstandards.com
license: GPL
vermagic: 2.6.33.3-rt19 SMP preempt mod_unload 486
depends:

That's odd how vermagic shows "2.6.33.3-rt19" when "uname -r" returns "2.6.33.3-l1-rt19"

Sorry, I have to go to work darn-it, but will check on the posts from there, and can continue to (hopefully) finish this off tonight !

Thanks again for all the help.

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 14, 2021, 06:38:26 AM
Hi MTCAT
... That's odd how vermagic shows "2.6.33.3-rt19" when "uname -r" returns "2.6.33.3-l1-rt19" ...
That's also why it returns  Invalid module format.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 14, 2021, 06:38:48 AM
Hi Rich,

Type corrected below in modinfo path, sorry again, that's a lot of sorry's for one morning !

"modinfo /home/tc/24dsi.linux.4.11.91.32.0/24dsi/driver/24dsi.ko" returns,

filename: /home/tc/24dsi.linux.4.11.91.32.0/24dsi/driver/24dsi.ko
description: 24DSI driver 4.11.91.32.0
author: General Standards Corporation, www.generalstandards.com
license: GPL
vermagic: 2.6.33.3-rt19 SMP preempt mod_unload 486
depends:

That's odd how vermagic shows "2.6.33.3-rt19" when "uname -r" returns "2.6.33.3-l1-rt19"

Sorry, I have to go to work darn-it, but will check on the posts from there, and can continue to (hopefully) finish this off tonight !

Thanks again for all the help.

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 14, 2021, 10:25:15 AM
Hi Rich,

Do you have any ideas why the GSC "start" script is seeing a different kernel version ?

As I see it, I have maybe two options;

1) Try inserting 24dsi.ko on the pre-built kernel again (2.6.33.3-l1-rt19) but using  modprobe with --force-vermagic option ?, reading up on that this morning, to tell modprobe to ignore kernel string mismatch ? Seems a bit disappointing to have to do that though ?

2) Try to make the DIY RT kernel (2.6.33.3-rt19-tinycore) behave properly as it already can build and insert the 24dsi.ko module properly, but the install has other issues still.

With respect to number (2), I don't know if you have the time to look through my "recipe" for the DIY RT kernel build (see reply 119), I'm pretty sure I used the wrong config file, so maybe that explains all the weirdness right off the bat, but, I'm wondering if you see any other mistakes in that list....I could try doing the DIY RT kernel again this weekend and see if we can get a better result ?

What do you think ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 14, 2021, 12:28:16 PM
Hi MTCAT
What does this return:
Code: [Select]
cat /lib/modules/2.6.33.3-l1-rt19/build/include/generated/utsrelease.h
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 14, 2021, 04:32:11 PM
Hi Rich,

That command returns

#define UTS_RELEASE "2.6.33.3-rt19"

Should be 2.6.33.3-l1-rt19 I guess...

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 14, 2021, 06:09:27 PM
Hi MTCAT
...  I'm pretty sure I used the wrong config file, ...
Did you use the config file you created for the RT kernel you built instead of this one:
http://repo.tinycorelinux.net/3.x/contrib/rt-kernel/src/config-2.6.33.3-l1-rt19

If that's the case, you may have to rerun Juanitos instructions:
http://forum.tinycorelinux.net/index.php/topic,24944.msg159183.html#msg159183

You could try correcting it in  /lib/modules/2.6.33.3-l1-rt19/build/include/generated/utsrelease.h , though I have no idea
if that will work.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 15, 2021, 07:57:58 AM
Hi Rich,

Sorry this is confusing, when I refer to using the wrong config file, I'm talking about the "DIY" RT build that I attempted some time ago. For that I used, "wget http://tinycorelinux.net/3/x/release/src/kernel/config-2.6.33.3-tinycore".

I think that was wrong, and maybe this is the source of all the problems with the "DIY" RT build ?

For the "pre-built" RT kernel, to make the patched RT module directory to compile against, and to make the symbolic link too, I did use the config file you correctly linked me to.

I just was thinking I should have used this "RT" config file you linked me to for the DIY RT kernel build too, I could try to do the "DIY" RT kernel build again with the correct config file, just not sure if there are other mistakes in my list (reply 119).

Will see if modprobe 24dsi.ko --force-vermagic works to load the driver in the pre-built RT kernel.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 15, 2021, 08:37:11 AM
Hi Rich,

The --force-vermagic doesn't seem to be a valid option with modprobe in TC3.8.4, and "-f" with insmod didn't work either, there's my big contribution :).

But, typing "sudo editor utsrelease.h", and changing "2.6.33.3-rt19" to "2.6.33.3-l1-rt19", and then running "sudo ./make_all clean" in ..../24dsi/driver, followed by "sudo ./make_all" worked !, the 24dsi.ko driver was built and inserted !, "lsmod" shows "24dsi" there, and I can now run the 24dsi test programs !

Since my RT patched linux-2.6.33.3 directory (what I'm symlinking too) is in home, the changes to utsrelease.h should be persistent too, so that's nice.

So this looks to be a valid work-around, thanks so much, do you think this is okay to go forward with though ?, or should I still try to re-do the DIY RT kernel ?, if the latter worked okay, would seem to be less hoops to jump through, "cleaner".

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 15, 2021, 11:51:29 AM
Hi MTCAT
... So this looks to be a valid work-around, thanks so much, do you think this is okay to go forward with though ? ...
Since the module loads and the test programs work it is probably OK.

Quote
, or should I still try to re-do the DIY RT kernel ?, if the latter worked okay, would seem to be less hoops to jump through, "cleaner".
I looked through the patch and config files and didn't see the  l1  part of the version string being added. I did see the patch
changing the  EXTRAVERSION =  line from  .3  to .3-rt19  in the top level  Makefile.  You can see it like this:
Code: [Select]
head /lib/modules/2.6.33.3-l1-rt19/build/Makefile
If you want to fix it, I would say just rebuild the headers like Juanito described:
http://forum.tinycorelinux.net/index.php/topic,24944.msg159183.html#msg159183
After you apply the patch, edit:
Code: [Select]
/home/tc/linux-2.6.33.3-patched/Makefileand change the  EXTRAVERSION =  line from  .3-rt19  to .3-l1-rt19

Then continue until you get to  make modules. Make sure the  Makefile  still has  EXTRAVERSION = .3-l1-rt19.
If all is good, run  make modules  and wait for it to complete.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 15, 2021, 11:54:39 AM
Hi MTCAT
When it's done, check that  utsrelease.h  contains the correct value.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 15, 2021, 01:45:20 PM
Hi Rich,

Thanks, maybe I'll leave my current working 2.6.33.3-l1-rt19 install as is, and on the next installation (I hope to build 5 or 6 of these loggers), I'll modify the Makefile as you suggest and see if that does the trick !

Thanks again, now I can make the squashfs extension and get the auto-loading going, get chrony fired up, and the last challenge will be to compile the data acquisition program and get it going !

Thanks,

David

Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 15, 2021, 02:46:55 PM
Hi Rich,

As you know, I unpacked linux-2.6.33.3-patched.tbz2 in /home/tc, does it make sense that having this big directory full of stuff would slow down booting up ?

Right now, after the empty backup file gets loaded, the little VortexDX3 takes quite a time (1 minute or so) to actually get into FLWM.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 15, 2021, 05:59:28 PM
Hi MTCAT
... does it make sense that having this big directory full of stuff would slow down booting up ? ...
Your home directory is persistent so nothing in it should be getting backed up. So the answer is it shouldn't slow you down.

What does this show:
Code: [Select]
cat /opt/.filetool.lst
Add the boot code  syslog  to your  extlinux.conf file. It goes on the line that contains the word  quiet.
After FLWM comes up:
Code: [Select]
cp /var/log/messages messages.txtAttach  messages.txt  to your next post.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 16, 2021, 01:06:10 PM
Hi Rich,

Thanks for the help, files2.txt contains the output of "cat /opt/.filetool.lst" and messages.txt contains the output from syslog.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 16, 2021, 02:17:54 PM
Hi MTCAT
You have  home  listed in  /opt/.filetool.lst.

Check your  extlinux.conf  file. Does the entry you are booting to have  home=UUID=...  similar to this:
Code: [Select]
LABEL realtime
KERNEL /boot/bzImageRT1
APPEND initrd=/boot/tinycoreRT1.gz quiet waitusb=5:UUID="622fa3dd-62bb-43fc-a837-43021b9b3c7f" tce=UUID="622fa3dd-62bb-43fc-a837-43021b9b3c7f" home=UUID="622fa3dd-62bb-43fc-a837-43021b9b3c7f" opt=UUID="622fa3dd-62bb-43fc-a837-43021b9b3c7f"

If it does, then after  FLWM  comes up, remove  home  from  /opt/.filetool.lst  and do this:
Code: [Select]
filetool.sh -bThen reboot and it should start faster.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 16, 2021, 05:07:53 PM
Hi Rich,

I do have persistent home and opt setup (as you showed me long ago) in extlinux.conf, so I removed "home" from /opt/.filetool.lst and ran "filetool.sh -b", unfortunately it didn't seem to help, seems to be something else, right before FLWM starts up I see some things flash by, will try to capture that on my iPhone.

And annoyingly, I can't load the 24dsi.ko module with modprobe anymore, I get an error, "modprobe: module 24dsi.ko not found in modules.dep".

I did run "sudo depmod -a" again but that didn't help, sorry about this, just never seems to go smoothly. I don't think the symbolic link has to be there for modprobe to load the 24dsi.ko module ?

"sudo insmod 24dsi.ko" still works to load the module though, so that's something at least....

I'm leery to nuke my already built and insertable module but maybe I just need to run "sudo ./make_all clean" and "sudo ./make_all" again ?

Party never ends, on power-down I'm also seeing a new message;

sd 0:0:0:0: [sda] Stopping disk ACPI: Preparing to enter system sleep state S5
Power down

Not sure why that popped up now...sorry for all these questions and problems, sda5 (128 GByte CF card) is not even mounted, so....

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 16, 2021, 05:16:25 PM
Hi MTCAT
... so I removed "home" from /opt/.filetool.lst and ran "filetool.sh -b", unfortunately it didn't seem to help, ...
Check  /opt/.filetool.lst  and make sure it still does not contain  home.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 16, 2021, 05:54:31 PM
Hi Rich,

"home" is indeed absent from /opt/.filetool.lst.

I do notice that there's some Adobe Flash related stuff, and Opera related stuff in /opt/.xfiletool.lst ?, I think those are both web browser related ?, which I certainly don't need, not sure if that's part of the problem.

By the way, for the fun of it, on the 2.6.33.3-l1-rt19 "pre-built" kernel, I also tried to do a "sudo depmod -a", "udevadm trigger" and "modinfo -d zram" and got nothing back, just a command prompt, no description returned, just like what happened with the DIY RT kernel, but there we were seeing "Ramzswap error" on boot-up, I don't see anything like that with the "pre-built" RT kernel on boot-up.

Also, past couple of power-downs, I'm not seeing the ACPI message anymore, not sure why it popped up in the first place.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 16, 2021, 06:03:17 PM
Hi MTCAT
... I do notice that there's some Adobe Flash related stuff, and Opera related stuff in /opt/.xfiletool.lst ?, ...
That's a generic list of stuff to exclude from the backup. Leave it as is.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 18, 2021, 12:21:57 PM
Hi Rich,

Okay, thanks will do. I'll continue playing around with the boot-up thing, regarding modprobe though, should I worry about it, or just use insmod ?

For modprobe, it seems I need to get I 24dsi.ko added into /lib/modules/2.6.33.3-l1.rt19/modules.dep.

Maybe I need to manually put 24dsi.ko in /lib/modules/2.6.33.3-l1-rt19/kernel, and then run "sudo depmod -a", to get 24dsi.ko added to /lib/modules/2.6.33.3-l1.rt19/modules.dep ??

Or, can I put a line in modules.dep myself, like, /lib/modules/2.6.33.3-l1-rt19/kernel/24dsi.ko ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 18, 2021, 01:55:45 PM
Hi MTCAT
I'm sure we covered this before. What did we do in the non real time kernel?

Did we add this to bootlocal.sh:
Code: [Select]
cp -a /home/tc/PATH/TO/24dsi.ko /lib/modules/2.6.33.3-l1-rt19/kernel
depmod -a

Or did we copy it to a directory under kernel and back it up:
Code: [Select]
sudo mkdir /lib/modules/2.6.33.3-l1-rt19/kernel/drivers/dsi
cp -a /home/tc/PATH/TO/24dsi.ko /lib/modules/2.6.33.3-l1-rt19/kernel/drivers/dsi
Then add  lib/modules/2.6.33.3-l1-rt19/kernel/drivers/dsi  to  /opt/.filetool.lst
Add  depmod -a  to  bootlocal.sh.
And run a backup:
Code: [Select]
filetool.sh -b
You might also want to review what changes we made to the  start  script and where you were calling it from.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 18, 2021, 02:04:04 PM
Hi MTCAT
... Or, can I put a line in modules.dep myself, like, /lib/modules/2.6.33.3-l1-rt19/kernel/24dsi.ko ?
No no no. That's what  depmod -a  is for.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 18, 2021, 05:55:01 PM
Hi Rich,

I'm probably wrong, but as I recall, manually typing "sudo modprobe 24dsi.ko" would work with the non-RT kernel, we only had issues with modprobe (with the non RT kernel) when we were trying to auto-load the 24dsi.ko module at boot time, that's when we had to comment out a few lines in the GSC start script (an if-fi structure that checked for the module existence - we commented that out).

So we kind of did your second option below, but only when we made the extension ? In /home/tc, we did "mkdir -p package/usr/local/lib/modules/2.6.33.3-tinycore/kernel/drivers/24dsi", and then;

cp 24dsi.linux.4.11.91.32.0/24dsi/lib/lib24dsi_api.so package/usr/local/lib
cp 24dsi.linux.4.11.91.32.0/24dsi/driver/24dsi.ko package/usr/local/lib/modules/2.6.33.3-tinycore/kernel/drivers/24dsi

And then we also copied the executable test programs into package/usr/local/bin with a big fancy command that copied files of the same name as the directory they are located in.
 
But that was all for making the extension I thought, and modprobe worked prior to that, at least manually ("sudo modprobe 24dsi.ko).

I'll study my "Where to build ADC module" printout (small textbook :) ) some more at work tomorrow, maybe I'm forgetting something.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 18, 2021, 06:24:42 PM
Hi MTCAT
Yeah, I remember that now. By making an extension out of those files some housekeeping gets handled for you just by
loading the extension.  tce-load  knows there's a kernel module present so it runs  depmod -a  and  udevadm trigger.
It also runs  ldconfig  so the system can find the library (lib24dsi_api.so).

Didn't we also copy the  start  script into the extension after making some changes? I seem to recall telling you to
alter the name (start_24dsi ?) because  start  was too generic.

If you copy the non real time extension to your  /home/tc  directory, you can unpack it and check what's inside:
Code: [Select]
unsquashfs -d NonRT ExtensionName-2.6.33.3.tczThe contents can be found in  NonRT.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 20, 2021, 11:19:19 AM
Hi Rich,

Finally had a chance to look at my printout, the only big difference I can see is that initially I was trying to build the 24dsi driver on the non RT install in /usr/src/linux/drivers, but then I switched over to building the driver in /home/tc, whereas in the RT case, I just started off building the driver in /home/tc.

Regarding the start program, we made a directory called ~/package/usr/local/bin and put it in there as "start24dsi", of course, to get "start24dsi" to work, we had to comment out those 5 lines in the script that checked for module existence.

So I don't think I'm any further ahead really, I still don't know why "sudo modprobe 24dsi" doesn't work in the RT case (no 24dsi link in modules.dep), yet modprobe did work (when typing it out manually) in the non-RT case, the regular non-modified GSC start script using "insmod" works though in the RT case...

But after loading the driver with "insmod", with the GSC "start" script,  on the RT kernel, I still don't see any 24dsi listing in modules.alias or modules.dep, the commands below return nothing, "grep 24dsi /lib/modules/2.6.33.3-l1-rt19/modules.dep", "grep 24dsi /lib/modules/2.6.33.3-l1-rt19/modules.alias", "modprobe -l 24dsi", "which start" "modinfo 24dsi.ko" does work, but only in when in the directory ~/24.dsi.linux.4.11.91.32.0/24dsi/driver.

Should I worry about getting modprobe to work on the RT install ?, or just make an extension this time using the GSC "start" script as-is, using insmod ?, it seems to work, not sure if "insmod" could be a problem in some way down the road ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 20, 2021, 03:08:33 PM
Hi Rich,

Check this out, "nonrt.txt" contains the output of "sudo find / -name 24dsi.ko" for the non RT kernel, note the last line in "nonrt.txt", how did that get there ? I'm assuming that's what we need and where we need it for modprobe ?

I also attached the same thing for the RT kernel, "rt1.txt" is before running "sudo ./start", and "rt2.txt" is after loading 24dsi.ko by running "sudo ./start", there's no difference and the driver file isn't where it's needed for modprobe ?

I'm not sure, hopefully this gives you some clues.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 20, 2021, 09:00:28 PM
Hi MTCAT
... But after loading the driver with "insmod", with the GSC "start" script,  on the RT kernel, I still don't see any 24dsi listing in modules.alias or modules.dep, ...
That's because  insmod  doesn't update  modules.alias  or  modules.dep , depmod  does.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 20, 2021, 09:45:10 PM
Hi MTCAT
Check this out, "nonrt.txt" contains the output of "sudo find / -name 24dsi.ko" for the non RT kernel, note the last line in "nonrt.txt", how did that get there ? ...
It came from the first line in that file:
Code: [Select]
/tmp/tcloop/24dsi-2.6.33.3-tinycore-686/usr/local/lib/modules/2.6.33.3-tinycore/kernel/drivers/24dsi/24dsi.koThat's the extension you made containing the driver, library, sample programs, and the start24dsi script.

Quote
...  I'm assuming that's what we need and where we need it for modprobe ? ...
Yes.

You need to package the driver, library, sample programs, and the start24dsi script.
These 2 posts covered that:
http://forum.tinycorelinux.net/index.php/topic,24795.msg157885.html#msg157885
http://forum.tinycorelinux.net/index.php/topic,24795.msg157950.html#msg157950
Change  2.6.33-tinycore  to  2.6.33.3-l1-rt19.
Copy the modified  start24dsi  script to  package/usr/local/bin.

This is where you finally got it working:
http://forum.tinycorelinux.net/index.php/topic,24795.msg158234.html#msg158234
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 21, 2021, 09:46:37 AM
Hi Rich,

Thanks, okay, I'll just go ahead and build the extension around the start24dsi script using modprobe, with that if-fi commented out, and hope that it all works, I'm just still wondering why modprobe isn't working with the RT install even when doing it manually, hopefully tce-load will take care of it all I guess ! Will give it a try.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 21, 2021, 12:13:38 PM
Hi MTCAT
modprobe should work if you:
Code: [Select]
sudo mkdir -p /usr/local/lib/modules/2.6.33.3-l1-rt19/kernel/drivers/dsi/
Code: [Select]
sudo cp -a 24dsi.ko  /usr/local/lib/modules/2.6.33.3-l1-rt19/kernel/drivers/dsi/
Code: [Select]
sudo depmod -a
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 22, 2021, 08:23:55 AM
Hi Rich,

Thanks alot, I'll see if I can get modprobe to work manually first before building the extension, hopefully today, thanks a lot.

Cheers,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 23, 2021, 02:06:16 PM
Hi Rich,

After following your instructions in reply 175 I was able to load the 24dsi module with modprobe !, thanks !

I then went ahead and made the extension file system with the modified start script, I got it all to work now ! !, auto-loading with the RT kernel !!, thanks again for all your help !

I did have some confusion with bootlocal.sh though, two things;

1) Are there actually two copies of bootlocal.sh ? One in /opt and another in /mnt/sdb1/opt ? I was initially editing the latter as that's what was in my notes, adding in the /usr/local/bin/start24dsi command to that file, but then I reverted to editing the one in /opt in the end ?
2) I was editing bootlocal.sh but then not doing a b/u, so on re-boot, the old bootlocal.sh (w/o the start24dsi command) would overwrite the new one !

I forgot about this need to do a b/u and was confused how my bootlocal.sh was getting changed when I'm using a persistent /opt !

Anyway, I did a b/u with "filetool,sh -b", but then re-named "mydata.tgz" to "rtmydata.tgz", to avoid what happened before, with the b/u overwriting my already persistent data, is that okay ?, I think we did that before, or would it be better to do the backup and then just use "norestore" boot code ?
 
The system is still a but flakey on boot-up though, it's spinning it's wheels doing something of which I'm not sure, I'm hoping it has something to do with chrony or gpsd, those packages are installed and being loaded, but I don't have the GPS hooked up yet with the RT kernel, so maybe that's giving the OS a problem, it's looking for the GPS that isn't there, and it does so until it times out or something. I may just try removing those extensions from onboot.lst and see if that changes anything.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 23, 2021, 07:22:35 PM
Hi MTCAT
... 1) Are there actually two copies of bootlocal.sh ? One in /opt and another in /mnt/sdb1/opt ? ...
No, there are not.  /mnt/sdb1/opt  is a persistent directory that resides on your storage device. That directory gets
mounted in the root file system as  /opt.

Quote
... 2) I was editing bootlocal.sh but then not doing a b/u, so on re-boot, the old bootlocal.sh (w/o the start24dsi command) would overwrite the new one !

I forgot about this need to do a b/u and was confused how my bootlocal.sh was getting changed when I'm using a persistent /opt ! ...
We covered this. Remove  opt  and  home  from  /opt/.filetool.lst.  Then do this:
Code: [Select]
sudo touch /tmp/dummy
sudo echo "tmp/dummy" >> /opt/.filetool.lst
filetool.sh -b
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 25, 2021, 01:31:42 PM
Hi Rich,

Thanks for your (incredible) patience...I'll do the empty file b/u method again, I won't ask again !

Now that I have the 24DSI driver being auto-loaded, I took a crack at using syslog again to see where the system is spinning it's wheels during boot, the only thing that jumps out at me is a 33 second time period between 20:01:05 and 20:01:38 (line 574 to 575) when root is running "/usr/local/bin/touch /etc/sysconfig/newmodules" , this happens right at the end of loading all the extensions from /mnt/sdb1/tce/optional (the 24DSI driver is the last extension to be loaded right now).

Attached the file here. Does that 33 second time period seem okay to you ? Root is creating an empty file in /etc/sysconfig called newmodules and that takes 33 seconds ?

I also removed the loading of ntp, chrony and gpsd extensions but that didn't speed up the boot process that I could tell.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 25, 2021, 02:35:25 PM
Hi MTCAT
I think these messages are printed when events are completed, not started:
Code: [Select]
May 25 20:01:05 (none) local2.notice sudo:       tc : TTY=unknown ; PWD=/mnt/sdb1/tce/optional ; USER=root ; COMMAND=/usr/local/bin/touch /etc/sysconfig/newmodules
May 25 20:01:38 (none) user.info kernel: EXT4-fs (sda5): mounted filesystem with ordered data mode
That would suggest that either sda5 takes a while to mount, or something else that does not generate a kernel event
is running between the above listed events.

The empty file /etc/sysconfig/newmodules is a flag that tce-load sets when booting to signal it has loaded an extension
containing kernel modules.

When tce-setup sees that flag, it runs  depmod  and  udevadm trigger  which might be responsible for at least some of
that time.

Or the delay might be caused by something else. ::)
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 28, 2021, 06:36:54 AM
Hi Rich,

One last kick at the can regarding this extra delay I'm seeing with the "pre-fab" RT kernel, if I wanted to try to "turn-off" the ethernet, would it be enough to comment out the call to eth0.sh in bootlocal.sh ?

I'm wondering if the delay has anything to do with the ethernet as in the "DIY" RT build you might recall I saw quite a long delay with this SIOCSIADDR or whatever thing, at least in that case, I saw an error echoed to screen, with the "pre-fab" RT build I'm not seeing that, but I just wonder if there's something quirky going on with the ethernet that's causing a delay.

Also, I did have strange behavior previously with the ethernet on the "pre-fab" RT build when downloading packages, I had to have another terminal open constantly pinging to keep the downloading of packages going, else it would stall, so maybe there's issues still with the ethernet.

Thought I would try just turning it off if possible, and see if it boots a lot faster, like 60 seconds faster !

The only other odd thing is the ACPI message I get at shutdown regarding the CF-Card entering sleep state S5.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 28, 2021, 07:33:03 AM
Hi MTCAT
...  if I wanted to try to "turn-off" the ethernet, would it be enough to comment out the call to eth0.sh in bootlocal.sh ? ...
Yes.

Quote
Also, I did have strange behavior previously with the ethernet on the "pre-fab" RT build when downloading packages, I had to have another terminal open constantly pinging to keep the downloading of packages going, else it would stall, so maybe there's issues still with the ethernet.
Didn't I already show you this:
http://forum.tinycorelinux.net/index.php/topic,24804.msg157651.html#msg157651
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 28, 2021, 07:42:37 AM
Hi Rich,

No difference in boot speed when commenting out call to eth0.sh in bootlocal.sh, for sure you did show me the keepalive thing, I just wasn't sure if that's a "band-aid" masking a deeper issue ?

I guess I'll proceed with getting chrony working (and tinierclock) so we can finally try to get the acquisition program going, I'm just hoping that there isn't something bad going on deeper down that will cause me to go back to square one so to speak (rebuild kernel).

Right now, it's about 90 seconds to boot up, I suppose that's not too bad, just driving me nuts though wondering what it's doing after it loads all the extensions !m and then sits there for what seems to be almost 60 seconds.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 28, 2021, 08:04:50 AM
Hi MTCAT
... Right now, it's about 90 seconds to boot up, I suppose that's not too bad, just driving me nuts though wondering what it's doing after it loads all the extensions ...
What's in your  tce/onboot.lst  file?
What's in your  /opt/bootlocal.sh  file?
What's in your  /opt/bootsync.sh  file?
What's in your  /opt/.filetool.lst  file?

Quote
... and then sits there for what seems to be almost 60 seconds.
So how long is it really?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on May 28, 2021, 09:18:54 AM
Hi Rich,

Attached the files here.

It takes about 46 seconds to load the extensions, and then the display sits there with Driver loaded: 24dsi on the screen for 67 seconds doing I'm not sure what ?

From power on, to FLWM being up, the whole boot process takes about 2 minutes, 35 seconds. Could take maybe 20 seconds off that as I'm doing a memory check on power up which also takes a bit of time, just need to turn that off in BIOS, but still would be looking at 2 minutes....

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on May 28, 2021, 11:08:39 AM
Hi MTCAT
... It takes about 46 seconds to load the extensions, ...
Clean up ypur onboot.lst file. First, remove these:
Code: [Select]
binutils.tcz
base-dev.tcz
bison.tcz
diffutils.tcz
file.tcz
findutils.tcz
flex.tcz
gawk.tcz
gcc.tcz
gperf.tcz
grep.tcz
m4.tcz
make.tcz
patch.tcz
pkg-config.tcz
sed.tcz
This is what  compiletc.tcz  loads. You are not currently compiling anything. If you don't have  compiletc.tcz  in your
tce/optional  directory, download it like this:
Code: [Select]
tce-load -w compiletc.tczIf you need to compile something in the future, you can add  compiletc.tcz  back to your  onboot.lst  file or load it manually:
Code: [Select]
tce-load -i compiletc.tcz
You can also remove the following:
Code: [Select]
syslinux.tcz         # Used to install syslinux boot loader on disk drives.
ncursesw-dev.tcz     # dev files are only needed when compiling.
bc-1.06.94.tcz      # I had you install that when you were compiling a kernel.
elfutils-dev.tcz     # dev files are only needed when compiling.
linux-headers-2.6.33.3-tinycore.tcz     # Only used when compiling.
squashfs-tools-4.x.tcz     # Only needed if you are packing/unpacking extensions.

It's possible you don't need to load  perl5.tcz  either.

Quote
... and then the display sits there with Driver loaded: 24dsi on the screen for 67 seconds doing I'm not sure what ? ...
That message came from the  start24dsi  script. It's the last message it echos right before it exits.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 04, 2021, 07:43:00 AM
Hi Rich,

I got chrony and gpsd working automatically (on boot-up) with the real time kernel, I had to make a new config file for chrony though, to make it work with GPS only (no online servers), I made a little script file "copychrony.sh" (/mnt/sdb1/opt) and put a command in bootlocal.sh (cp /home/tc/newchrony.conf /usr/local/etc/chrony/chrony.conf) and then put a file in /home/tc/.X.d called "start_time_sync" which contains the commands to start chrony and gpsd, it seems to work ! ! I think there are more tweaks that I could do to chrony, maybe to get it to work faster, but at least it's working on it's own, so that's exciting.

So, we're finally at the stage to start thinking about getting the data acquisition program, not sure what would be the place to start on that, just try to compile it and see what happens ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 04, 2021, 07:50:59 AM
Hi MTCAT
Does the currently compiled data acquisition program run as is?
Did the data acquisition program come with a  Make  file or a build script?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 04, 2021, 08:11:48 AM
Hi Rich,

I have two working instruments with the acquisition program on it, with that, there's an icon on the Slackware desktop called "MT Logger" and I just right click on that and hit "Execute" and up-starts the acquisition program.

I found a bunch of different b/u copies of the acquisition program, here's a Makefile from one of those b/u directories.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 04, 2021, 08:41:06 AM
Hi MTCAT
It looks like the following are required to build it:
compiletc.tcz
squashfs-tools-4.x.tcz
gtk2-dev.tcz
fftw-dev.tcz

and possibly:
libglade-dev.tcz

You might also want to change this in the  Make  file:
From:
Code: [Select]
@cp $(PROJECTNAME) ~/binTo:
Code: [Select]
@cp $(PROJECTNAME) ~/.local/bin
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 04, 2021, 09:13:33 AM
Hi Rich,

Thanks a lot, I'll download those extra extensions and give "sudo ./make all" a try, glad to see there's a glade library for flwm !

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 04, 2021, 11:34:40 AM
Hi Rich,

Well, after downloading the extra packages, and changing the copy path in the Makefile, the mt_logger program seems to have compiled just fine !, but the mt_adc program had a problem, it couldn't find two header files, "24dsi_utils.h" and "24dsi_dsl.h", I did a "find / -name" and found that both of these files are in "/home/tc/24dsi.linux.4.11.91.32.0/24dsi/include", should I just maybe copy them right into the same directory as the Makefile(s) ?

Also, for both mt_logger and mt_adc  I had to run "sudo make all", for some reason "sudo ./make all" didn't work.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 04, 2021, 12:41:18 PM
Hi MTCAT
In the make file, after the line that begins with  CFLAGS= , try adding this line:
Code: [Select]
CFLAGS+= -I/home/tc/24dsi.linux.4.11.91.32.0/24dsi/include
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 04, 2021, 12:59:58 PM
Hi Rich,

Thank you so much, I think that worked to find those header files, only now the make script for mt_adc can't find "24dsi_utils.a", "24dsi_dsl.a" and "gsc_utils.a", these guys are all located in /home/tc/24dsi.linux.4.11.91.32.0/24dsi/lib", can I add another CFLAGS line ?, like CFLAGS++= -I/home/24dsi.linux.4.11.91.32.0/24dsi/lib ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 04, 2021, 01:17:17 PM
Hi MTCAT
Try adding a second  CFLAGS+=  line:
Code: [Select]
CFLAGS+= -L/home/tc/24dsi.linux.4.11.91.32.0/24dsi/lib
By the way, I almost copy/pasted this path from your post:
... like CFLAGS++= -I/home/24dsi.linux.4.11.91.32.0/24dsi/lib ?
You left out the  tc  from  /home/tc/24dsi......  That omission caused us problems once before :o.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 04, 2021, 02:29:07 PM
Hi Rich,

Thanks, gee, I did that again, thanks for catching that. I added the second CFLAGS line but it didn't help, I've attached the Makefile here for you, it's the "LIBS= ../" line that's the problem I think, the Makefile as attached still gives me an error;

gcc: ../lib/24dsi_utils.a: No such file or directory
gcc: ../lib/24dsi_dsl.a: No such file or directory
gcc: ../lib/gsc_utils.a: No such file or directory
make: ***[mt_adc] Error 1

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 04, 2021, 04:57:34 PM
Hi MTCAT
Try changing the  LIBS =  line to this:
Code: [Select]
LIBS = 24dsi_utils.a 24dsi_dsl.a gsc_utils.a
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 04, 2021, 05:09:19 PM
Hi Rich,

Thanks for the help but darn-it that didn't work, I suppose it would be bad programming style, but could I just put the whole path in there, like

LIBS = /home/tc/24dsi.linux.4.11.92.32.0/24dsi/lib/24dsi_utils.a, and so on ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 04, 2021, 05:12:17 PM
Hi MTCAT
Sure, give that a try.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 05, 2021, 11:53:30 AM
Hi Rich,

I made a "lib" directory under /home/tc/mt_adc and pointed the Makefile there for the library files, so that worked (make can find the xxxx.a files now), but now there's a bunch of errors in three functions contained in "adc_control.c", one function called "open_adc", another function called "close_adc", and another called "read_adc", for each function, I get a bunch of errors like,

adc_control.c(.text+0xe): undefined reference to gsc_count_boards

And so on, I wonder if this is where using a different driver (4.11.91.32) is now the issue, perhaps the function names have changed with the new driver ? Need to "update" all the function calls in adc_control.c to the newer named versions ?

And then there's some errors too like the below,

/home/tc/mt_adc/lib/24dsi_utils.a: In function `dsi_close_util':
(.text+0x26): undefined reference to `dsi_close'

And so  on with a bunch of others for 24dsi_utils.a (dsi_init, dsi_ioctl, etc, et) and 24dsi_dsl.a (dsi_close, dsi_init, dsi_ioctl, dsi_open, dsi_read)

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 08, 2021, 06:45:32 AM
Hi Rich,

Not totally sure yet, still talking with GSC about it, but looks like I will need to build an older version of the 24DSI driver (3.17.52) to get my data acquisition program to work "as-is", either that or modify the acquisition program for the new function calls in the 4.11.92.32.0 driver....

Thanks,

David

Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 08, 2021, 01:40:18 PM
Hi MTCAT
... adc_control.c(.text+0xe): undefined reference to gsc_count_boards ...
I don't see  gsc_count_boards  defined anywhere in the driver source files.

Quote
... and 24dsi_dsl.a (dsi_close, dsi_init, dsi_ioctl, dsi_open, dsi_read)
I see them defined in the  dsi/api  directory in  close.c, init.c, ioctl.c, open.c, and read.c.
According to the  makefile , it creates a  dsi/lib/lib24dsi_api.so  file.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 08, 2021, 02:56:37 PM
Hi Rich,

Thanks for the help, I heard back from GSC and the gsc_count_boards function is not present in the new driver, they also said something to the effect that even if I did get the acquisition code to compile, the API has changed such that the acquisition program probably wouldn't work properly. GSC recommends building and using the older 3.17.52 driver, which is what my (former) colleague used (I think anyway).

Unfortunately I don't have the tar file, but I do have the expanded directory structure on my two working receivers, so I think I just need to copy that over to the VortexDX3 and do a "sudo make_all clean" to remove all remnants of the old build, and then a "sudo make_all", make a new extension, and then hopefully the existing "start_24dsi" script will still work ?, or, if I can stop being lazy, just do the same mod's (comment out the if-fi section checking for module existence) and then load up this old driver.

Then hopefully the acquisition program will be happy, fingers crossed. I'm doing the GPS/PPS a bit differently though, with both the NMEA string and PPS over the serial port (ttyS0), whereas before, only the PPS was over the serial port, and NMEA over USB, bso hopefully I just need to change ttyACMO to ttySO for the NMEA string (to get co-ordinate info), so that should be easy hopefully (famous last words).

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 08, 2021, 05:27:14 PM
Hi MTCAT
... Unfortunately I don't have the tar file, but I do have the expanded directory structure on my two working receivers, ...
So  tar  up the directory:
Code: [Select]
tar -czf 24dsi.linux.3.17.52.tar.gz DirectoryName
Quote
... and do a "sudo make_all clean" to remove all remnants of the old build, and then a "sudo make_all", ...
Shouldn't that be:
Code: [Select]
sudo make clean
sudo make all

Quote
... or, if I can stop being lazy, just do the same mod's (comment out the if-fi section checking for module existence) and then load up this old driver. ...
If you are talking about loading the old precompiled driver, the version string won't match the kernel, so it won't load.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 12, 2021, 09:14:05 AM
Hi Rich,

I removed the loading of the 4.11.91.32.0 24dsi driver from onboot.lst and then commented out the call to start_24dsi in bootlocal.sh so that I can try to build and install the old 3.17.52.0 driver.

I took all the old driver files off one of my existing receivers and just did a "cp -r" command to copy everything onto the VortexDX3, in /home/tc/24dsi.linux.3.17.52.0

I then changed directory to /home/tc/24dsi.linux.3.17.52.0/24dsi and typed "sudo ./make_all clean" to remove the old build, followed by "sudo ./make_all" .

Unfortunately, I got an error as below

driver===========================================
make:*** /lib/modules/2.6.33.3-l1-rt19/build: No such file or directory. Stop.
make:*** [24dsi.ko] Error 2
Driver loading: 24dsi...
---> ERROR <--- module file does not exist: /home/tc/24dsi.linux.3.17.52.9/24dsi/driver/./24dsi.ko

Would it be enough to simply make a "build" directory, in /lib/modules/2.6.33.3-l1-rt19/build ? Sorry for all the problems with building the 24DSI driver, nothing seems to go "by the book" with the 24DSI.

I attached the "make_all" script for the old 3.17.52.0 drier here as well, it's quite a bit different, seems to dart in and out of the various sub-directories running make commands in each sub-directory, and fails in the driver directory.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 12, 2021, 09:20:56 AM
Hi Rich,

Sorry for the type in the "--->ERROR<---- " line, it should be "......3.17.52.0/24dsi/driver/./24dsi.ko"

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 12, 2021, 01:03:58 PM
Hi Rich,

Sorry for all the chatter, I forgot that I had to make a symbolic link "sudo ln -s /home/tc/linux-2.6.33.3 /lib/modules/2.6.33.3-l1-rt19/build" first, for the driver to build against as Juanito would say, I did that and was able to build the 3.17.52.0 driver just fine !, and it also loaded (inserted) successfully !

Whew, that's a relief, now I can spend the rest of the weekend building the extension and seeing how this version of the "start" program behaves when trying to auto-load, I assume I will need to comment out the "if fi" structure that checks for module existence again, and swap out "insmod" with "modprobe" as well !

I attached the old version (for the 3.17.52.0 driver) of the "start" program here, it's a bit different I think ?

And then, can finally try compiling and running the acquisition code with the old driver !

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 13, 2021, 11:30:44 AM
Hi Rich,

I got the old driver to auto-install ! !, doing the same mod's we did before to the "start" script ! ! Even better, I was able to get both mt_logger and mt_adc to properly compile too ! !, I had to copy a bunch more header files into place though, but, wow, this is awesome...

In Slackware, when I start the acquisition program, there's an icon on the desktop labeled "MT Logger", I right click on it and then hit execute, after doing that two terminal windows pop up, one for MT_Logger, and another for MT_adc, and then lastly a graphical time series viewer/acquisition setup window pops up over top of the two terminal windows.

I think I need to somehow find out if that icon on the desktop in Slackware is linked to a script file, and then see what's in the script file, in order to properly start the acquisition program ?

I think mt_logger starts first, and then mt_adc starts second.

So I tried to start mt_logger in TinyCore by typing "sudo ./mt_logger" but it (mt_logger) couldn't find a config file it needs, it was looking for "/home/pete/config/mt_logger.conf" and then got a segmentation fault cause of course that directory and file don't exist (I can get the config file off one of my working receivers though).

I looked in a program called "setup.c" in the mt_logger directory and a structure is defined in there, in which a character variable called "filename" exists, but that path above (to the config file) is never shown anywhere, anyway, I just have to figure out some of these types of things and hopefully can get the acquisition code going, this is very exciting, would not have been able to get anywhere this close w/o all your help, thanks a lot, you'll probably be relieved to be almost done with me ! !

Hopefully there aren't any "gotcha's" lurking ahead w.r.t graphical windows type of stuff, I guess we'll see.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 13, 2021, 01:52:49 PM
Hi MTCAT
... (mt_logger) couldn't find a config file it needs, it was looking for "/home/pete/config/mt_logger.conf" ...
Are you sure there wasn't a period in front of  config:
Code: [Select]
/home/pete/.config/mt_logger.confYou may get away with sloppy spelling, capitalization, and punctuation in school, but try that with Linux and you'll get schooled.

Quote
... I looked in a program called "setup.c" in the mt_logger directory and a structure is defined in there, in which a character variable called "filename" exists, but that path above (to the config file) is never shown anywhere, anyway, ...
That setup.c could be a stand alone program that creates the config file, or it could be part of  mt_logger  and/or  mt_adc.
Do  mt_logger  or  mt_adc  take any command line arguments:
Code: [Select]
./mt_adc  --helpor
Code: [Select]
./mt_adc  -h
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 13, 2021, 03:03:52 PM
Hi Rich,

No, there was no "." in front of config, the path to the config file that mt_logger was looking for was/is  "/home/pete/config/mt_logger.conf", weird thing is, in my "user manual" for running my existing systems, I have written down that mt_logger.conf should be in home/config, so a different path.... There's some filter settings and other things that can be changed in mt_logger.conf

I also found written down that there is a script called "start_logger.sh" in "~/bin", this may be what gets run when I right click on the desktop icon (in Slackware) and select "Execute" ? This script is less intended to poke around in I think, but there is a switch inside it to turn file compression on and off, but it's the mt_logger.conf file that's more commonly played around in.

I need to poke around my existing fully assembled receiver and see what I can find there...I need to find out how to point mt_logger to a different place, to load mt_logger.conf, among other things, like using the "start_logger.sh" script too.

I'm sure I will continue to need schooling in the world of Linux for quite some time  ;D, probably forever, lol.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 13, 2021, 03:11:42 PM
Hi Rich,

Sorry, forgot to add, no, unfortunately there is no help facility in the mt_logger or mt_adc "home-brew" programs.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 13, 2021, 06:38:32 PM
Hi MTCAT
... weird thing is, in my "user manual" for running my existing systems, I have written down that mt_logger.conf should be in home/config, ...
As far as I know, the home directory should only contain user subdirectories:
https://www.linuxnix.com/linux-directory-structure-home-root-folders/

Quote
... I also found written down that there is a script called "start_logger.sh" in "~/bin", ...
Tinycore does not implement  ~/bin.  If you want something user specific that is included in the path, place it in  ~/.local/bin.

Code: [Select]
... /home/pete/config/mt_logger.conf ... Was pete the author of the software?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 14, 2021, 05:58:28 AM
Hi Rich,

Thanks for the help and advice, yes, Peter was the fellow who wrote the acquisition software and set-up the digital portion (hardware and software) of my two original receivers.

Hopefully I can poke around on one of my receivers in the next few days, I still can't find any reference to "/home/pete/config/mt_logger.conf" in any of the C programs yet !
Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 14, 2021, 08:30:34 AM
Hi MTCAT
... I still can't find any reference to "/home/pete/config/mt_logger.conf" ...
It could be in a header file, a makefile, a configure script, etc. Try this:
Code: [Select]
sudo grep -ri pete ~/*This will search every file in your home directory for the text  pete  regardless of case. This also presumes the source
code for the acquisition software is somewhere in your home directory.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 19, 2021, 02:18:01 PM
Hi Rich,

Thanks for the help, after looking through all the C (C++ ?) programs in the two directories I was able to find a few "#define xxxx" lines, like, "#define CFGFILE /home/pete/config/mt_adc.conf", so that's great, I just need to change those to what I want, and put the config files in those spots, and that should be okay.

I also need to change ttyUSBO to ttyS0 inside "gps.c", so that's an easy one as well.

I do need to take care that my data directory (where data is stored) right now is "/home/data", that would currently be written to the pen drive ! (slow), I want it to rather go to the CF-card, I'm wondering if "/mnt/sda5/data" or something like that would work, or, perhaps in the future, maybe I should just install TC completely onto the CF card....

Just a couple of other things though, Peter defines a shared memory segment of 256 MBytes for the logger program, I'm not sure yet if he has this in a script file or something but in the comments of a C (C++) program called "shared_mem.c" he says that we can do this by typing "echo 268435456 > /proc/sys/kernel/shmmax".

I guess I could run that (or similar) with a script file called out of bootlocal.sh ?, or perhaps I could just edit sysctl.conf and make that change in there permanently ?, assuming that file exists in Tinycore ?

Then the only other thing relates to the good old PPS, I'm planning on using a TTL level PPS (0 to 3.3 V) whereas the current setup uses an RS232 level PPS which has been divided down from  it's original ~ +6 to - 5 V range to a roughly symmetrical +/- 1.3 V (kind of messily done with two voltage dividers).

Inside a program called pps_stats.c, Peter has a bunch of "cumulative distribution functions" (in lookup tables) that he's using to predict where the zero crossing (defined as the midpoint of the low-high transition) of the PPS signal is (I think anyway), but I'm not really sure, C (C++ ?) code always looks odd to me, so, I'm  not sure, if I redefine the PPS waveform do I need to collect a bunch of these "new" PPS waveforms, make a probability amplitude distribution and then integrate it to get a new cdf ?? I don't know if that's what Peter really did, I don't recall him ever talking about that, only that we need to know the +/- levels of the PPS signal.

I'm puzzled on this one, and have always been a bit puzzled as to what Peter was/is doing with the PPS signal, I even recall him saying something about adding extra samples if need be (from where ??-talk about distorting your measured data - -making up new samples ??) to account for a slightly off sample rate. Peter uses the PPS to measure the actual sample rate of the 24DSI, maybe it's 199999.79 Hz in reality when asking for 200 kHz for example....

A colleague of mine at work thinks Peter may be compensating for the effect of the FIR filter delay (in the delta-sigma ADC) on the PPS, but I don't know about that, why do it for the PPS channel and not for the other channels, and also, if all channels are delayed the same (same FIR filter applied to all channels), then there's nothing lost in terms of relative timing on the data, so I'm not sure about that one.

My confusion around the PPS continues. I was wondering if I could simply use a 2 nano-second sampling oscilliscope (8 bit voltage res though) to "hyper-fine" define the PPS signal that way, and then simply interpolate PPS zero-cross location based on the "sparse" 200 kHz sampled PPS waveform, I think that's what Peter is doing with the cdf, but I don't really understand how he's doing it, so I'm not sure.. I guess I don't want to re-write Peter's program either though, as I'm pretty much clueless with C (C++).

I could send you the pps_stats.c code for you t look at, if you have the time, and see what you think, it's only about 20 lines of actual code, the rest is tables. Peter says he's "compensating for lag between the A/D and the PPS", so that does kind of sound like what my co-worker suggested.

But I do recall Peter saying some odd things about ways of finding the zero-cross of the PPS with sparsely sampled (200 kHz) data too, so, going in circles on this one.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 19, 2021, 06:30:55 PM
Hi MTCAT
... I was able to find a few "#define xxxx" lines, like, "#define CFGFILE /home/pete/config/mt_adc.conf", so that's great, I just need to change those to what I want, and put the config files in those spots, and that should be okay. ...
A better solution would be to not hard code a particular user, but have the program default to the home directory of
the user currently running the program:
Code: [Select]
char cfgfile[256];
snprintf(cfgfile, 256, "%s/.config/mt_adc.conf", getenv("HOME"));
If user  pete  runs the program, charcter array  cfgfile  gets set to  /home/pete/.config/mt_adc.conf.
If user  tc  runs the program, charcter array  cfgfile  gets set to  /home/tc/.config/mt_adc.conf.

That way it gets set at run time, not compile time.

Quote
... I want it to rather go to the CF-card, I'm wondering if "/mnt/sda5/data" or something like that would work ...
You could move  /home/data  and replace it with a link to  /mnt/sda5/data:
Code: [Select]
mv /home/data /home/dataold
Code: [Select]
mkdir /mnt/sda5/data
Code: [Select]
ln -s /mnt/sda5/data /home/dataMake sure  sda5  is mounted at the time. Be aware, devices such as  sda  can move depending on what devices are
connected when you boot. Identifying devices by UUID is safer.

Quote
... but in the comments of a C (C++) program called "shared_mem.c" he says that we can do this by typing "echo 268435456 > /proc/sys/kernel/shmmax". ...
Without seeing what else is around the comments (and actual comments) I have no context and can't respond to that.

Quote
... I could send you the pps_stats.c code for you t look at, if you have the time, and see what you think, it's only about 20 lines of actual code, the rest is tables. ...
Sure, I could take a peek at it.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 20, 2021, 12:57:12 PM
Hi Rich,

Thanks a lot, I like the idea of having the config file location being set "on the fly", that's an awesome idea I would say, thanks for that code, thanks as well for the idea to make a symbolic link, is the "mv /home/data /home/dataold" done just to make a backup copy though ?

I attached three programs here; "shared_mem.c" so you can see the shared memory segment stuff, and then "process_adc.c" and "pps_stats.c", inside "process_adc.c" around line 303 is where some PPS related stuff begins to happen, and then inside "pps_stats.c" is where all the "cumulative distribution functions" (CDF) are tabled and a bit of code is located as well.

Right now, we're using the U-Blox 200 kHz sample rate with 40 kHz filter table, Peter has a lot of extra tables in there that could be cleaned out, for example, I had an "ORCA GS101" GPS device in the past that we were using initially, it's quite nice but also quite expensive (~ 1500 bucks I think), so we switched over to the U-Blox's (~ 300 bucks) which work very well. The ORCA was really used more for my old Windows XP laptops that were used with my version 1 receiver.

But, if I redefine the PPS signal to be a 0 to 3.3 V TTL level (which would be similar to the ORCA actually --->maybe I could even just use the ORCA CDF tables...) then I'm wondering if I need to redo the CDF table for a U-Blox 200 kHz, 40 kHz filter, TTL level PPS.

Also, in "process_adc.c", there's two functions "pps_vmin()" (line 394) and "pps_vmax()" (line 402) that would need to be tweaked for the new signal amplitude of the U-Blox TTL output, which is about 0 V low and 3.3 V high, although I need to measure that to find out what the levels are more accurately.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 20, 2021, 01:04:38 PM
Hi Rich,

Re: the symbolic link question, thinking about it, I suppose we can't make a symbolic link with /home/data pointing to /mnt/sda5/data if /home/data also exists on the pen drive, makes sense that the OS would be confused to say the least.....hopefully I answered my own question, at least approximately correctly !

Thanks,

David 
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 20, 2021, 01:05:51 PM
Hi MTCAT
...  is the "mv /home/data /home/dataold" done just to make a backup copy though ? ...
I don't know whether  /home/data  is a file or a directory ( I presume directory). Either way, I didn't want to give
instructions that might destroy it. So  /home/data  gets renamed to  /home/dataold  to serve as a backup.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 20, 2021, 01:14:33 PM
Hi MTCAT
... I suppose we can't make a symbolic link with /home/data pointing to /mnt/sda5/data if /home/data also exists on the pen drive ...
I don't recommend it. However, you can alter the link later on if you need to, for example:
Code: [Select]
unlink /home/data
link -s /home/dataold /home/data
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 20, 2021, 08:32:32 PM
Hi MTCAT
...
Quote
... but in the comments of a C (C++) program called "shared_mem.c" he says that we can do this by typing "echo 268435456 > /proc/sys/kernel/shmmax". ...
Without seeing what else is around the comments (and actual comments) I have no context and can't respond to that. ...
That is absolutely not what he said. The kernel can limit the amount of shared memory that can be allocated. If
shared_mem.c  tries to allocate more shared memory than the kernel will allow, you may have to raise the limit using
that command. You can check what the limit is with this command:
Code: [Select]
tc@E310:~$ cat /proc/sys/kernel/shmmax
4278190079
tc@E310:~$

... Also, in "process_adc.c", there's two functions "pps_vmin()" (line 394) and "pps_vmax()" (line 402) that would need to be tweaked for the new signal amplitude of the U-Blox TTL output, which is about 0 V low and 3.3 V high, although I need to measure that to find out what the levels are more accurately.
Don't waste your time. Those levels will change with temperature and supply voltage variations. Uncomment the first
double and comment the second double in both functions like this:
Code: [Select]
static double pps_vmin()
{
    /* the pps signal swings between .136 and 3.460 volts
     */
     static double v_min = 0.137 ; // orca
//     static double v_min = -1.310 ; // ublox with resister divider
     return( v_min ) ;
}

Quote
... But, if I redefine the PPS signal to be a 0 to 3.3 V TTL level (which would be similar to the ORCA actually --->maybe I could even just use the ORCA CDF tables...) then I'm wondering if I need to redo the CDF table for a U-Blox 200 kHz, 40 kHz filter, TTL level PPS. ...
Do you know how those tables were created?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 21, 2021, 07:40:54 AM
Hi Rich,

Thanks for the help, I'll check to see what my shmmax is set to and go from there.

It could be a "fools hope" as you say, but I think it's fairly important that we know the mid-point of the high-low PPS signal, that's what Peter defines as the start of a second (I think anyway). But maybe using the ORCA TTL levels would be good enough as you suggest, it should be quite close to the U-Blox TTL level output (0 to 3.3 V).

No, unfortunately I don't know how Peter generated or obtained these CDF tables, from what little I know of statistics I think we could make one by collecting a nice long chunk of PPS data (ten minutes maybe ??), and then make a finely binned histogram plot, divide each bin in the histogram plot by the total number of samples (to obtain a probability amplitude distribution), and then lastly integrate that to get the CDF. But I have no recollection of Peter ever discussing this with me, so I'm not sure if this is what he actually did.....

I suppose I could try this process on the existing fully assembled receiver (U-Blox RS232 PPS that has been knocked down with a voltage divider) and see if I can ~ reproduce the CDF table for that case...Or just use the ORCA settings....

Can you hazard a guess at what the "lag" value is that Peter is returning out of pps_stats.c ? I'm guessing it's the interpolated location of the PPS "zero-crossing" (midpoint of the low to high transition), but I'm not totally sure, the CDF's are the probability that a function takes on a value less than or equal to some value, so I'm not sure how that works better to obtain zero crossings, or if Peter is just being overcomplicated ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 21, 2021, 08:03:06 PM
Hi MTCAT
... It could be a "fools hope" as you say, but I think it's fairly important that we know the mid-point of the high-low PPS signal, that's what Peter defines as the start of a second (I think anyway). ...
It sounds like he's taking measurements to detect when the signal makes a transition. It's not important that you sync
to the exact center of the transition, just that you always sync to the same level during the transition.

Quote
... I suppose I could try this process on the existing fully assembled receiver (U-Blox RS232 PPS that has been knocked down with a voltage divider) and see if I can ~ reproduce the CDF table for that case...Or just use the ORCA settings....
If you have TTL levels, go with the ORCA settings.

Quote
... Can you hazard a guess at what the "lag" value is that Peter is returning out of pps_stats.c ? ...
He returns a decimal value between 0 and 100.

This shows up at the end of  process_adc.c:
Code: [Select]
double process_precise_sample_rate()
{
    /* measure the rate by looking at last two pps crossings and lags
     *
     * the nominal sample time between the crossings (1 second) is adjusted
     * by the difference in lags of the two scans.
     *
     * lag is defined at a percent (0:100) of the sample interval (dt)
     */
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 23, 2021, 11:49:18 AM
Hi Rich,

I think you're right, using the ORCA defined table (for TTL) is the way to go, I may have figured out how Peter generated the tables but I'm not 100 percent sure, so...., probably just easiest/best to move forward with the ORCA table. Hopefully on Friday I can make some headway on getting the config files sorted out, and changing the code to use the ORCA table, re-compiling and try starting the acquisition program again, see what happens...

Down the road, I wonder if we could improve the zero-cross location by doing a super-fast sampling of the PPS waveform (1 GHz sample rate with a storage oscilloscope), even though it's only 8 bit voltage resolution, although I think there's a high resolution mode as well which gives 12 bits resolution, perhaps if I averaged enough cycles, I could get a good definition of the PPS waveform ? With that, we would have anywhere from 5,000 (200 kHz) to 10,000 (100 kHz) extra points between each 24DSI sampled point on the PPS waveform, I think we could find quite accurately where the zero-cross lies between two 24DSI sample points that bracket the zero-cross, this might be better, and a lot simpler. But I would have to collect a bunch of PPS waveforms and then average them, and of course modify the nasty (in my opinion) C++ code.

Besides the 1 GHz oscilloscope, I also have a 16 bit, 1 MHz, A-D (PDAQ 3005), and a 12 bit, 1 MHz A-D (Wavebook 512), but that would only be 5 (or 10) extra points between 24DSI samples.

Can save that for later though maybe..

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 23, 2021, 03:06:08 PM
Hi Rich,

As usual, egg on my face, I thought I figured out how Peter was making the tables but was totally wrong, I found another directory called "mt_timing" with some more programs in it, attached them both here, it looks like he generates a bunch of data (log) files with PPS timing info in them which then get loaded and analyzed to make the tables, but, I don't think I have the program that does the actual analysis (??), but maybe some insight there for us regarding the table generation. There's a bunch more data files, etc., in the "mt_timing" directory but size limits won't let me post it all. Unfortunately, Peter uses the same name (pps_stats.c) for two programs that are very different, he did the same thing with shared_mem.c, makes things a bit confusing, at least for me.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 23, 2021, 08:49:00 PM
Hi MTCAT
... Down the road, I wonder if we could improve the zero-cross location by doing a super-fast sampling of the PPS waveform ...
It's not a zero crossing, it's a threshold crossing. And it's not a hardware threshold, it's a software threshold. That means
the threshold is basically whatever you say it is. In pps_stats.c Pete defines it as:
Code: [Select]
(Vhi-Vlo)/2but there's nothing that says it can't be:
Code: [Select]
(Vhi-Vlo) * 6/10or:
Code: [Select]
(Vhi-Vlo) * 4/10
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 25, 2021, 09:15:04 AM
Hi Rich,

Thanks for the help, just getting going on this today,  for me,

cat /proc/sys/kernel/shmmax
33554432

Divide that by 1024 and we get 32,768. So I need to increase this to 256 MBytes, I did a "sudo find -name sysctl.conf" but nothing came up. I also tried "sudo echo 268435456 > /proc/sys/kernel/shmmax" but I got an error;

sh: can't create /proc/sys/kernel/shmmax: Permission denied

One other command I had written down, which I didn't try was, "sysctl -w kernel.shmmax=<value", maybe this is the correct way to do it ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: patrikg on June 25, 2021, 10:47:48 AM
When piping with sudo i don't think it work.
You have to use something like this.

Code: (bash) [Select]
sudo sh -c 'echo 268435456 > /proc/sys/kernel/shmmax'
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 25, 2021, 10:48:17 AM
Hi MTCAT
Try it like this:
Code: [Select]
echo 268435456 | sudo tee /proc/sys/kernel/shmmax
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 25, 2021, 10:57:42 AM
Hi MTCAT
... One other command I had written down, which I didn't try was, "sysctl -w kernel.shmmax=<value",  ...
Yes, you could also do this:
Code: [Select]
sudo sysctl -w kernel.shmmax=268435456
Or you can do it like patrikg suggested. Lots of ways to get the same result in Linux.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 25, 2021, 11:46:30 AM
Hi Rich and partikg,

Thanks a lot, I will try one (or more ) of those commands, I will test as well if this needs to be done on every boot-up, if so, I can just make a little script and then call it out of bootlocal.sh

One other complication, right now my CF-card (sda5) has Lubuntu (16.??) on it, so when I mounted it, and tried to "mkdir /mnt/sda5/data", I was denied permission as all the directories and file on sda5 appear to be "root" ownership. I could either wipe Lubuntu off the CF-card completely (by reformatting ?) or just "sudo mkdir /mnt/sda5/data" followed by "chown tc /mnt/sda5/data", I think that would be okay ? I assume I'll be running the acquisition program as a regular "tc" user.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 25, 2021, 12:05:37 PM
Hi MTCAT
Code: [Select]
sudo mkdir /mnt/sda5/data
sudo chown tc:staff /mnt/sda5/data
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 25, 2021, 06:26:44 PM
Hi Rich,

I added a couple more script calls to boolocal.sh, one to increase shmmax to 256 MBytes, and another to mount sda5, so that's cool, working automatically now at boot up, forgot again that I had to make the scripts executable (chmod 775 .....)

I also got all the config files off one of my existing receivers and found the startup-script linked to the desktop icon, attached it here.

It looks like the code on my existing (working receiver) is a bit different from the mt_logger and mt_adc code I previously compiled on the DX3, which I got from a "'home/pete/backup" directory (maybe was an old b/u) but I think I'll just keep taking "baby-steps" and proceed with playing around with the acquisition code that's on the DX3 (from the old b/u), what do you think ?

I only briefly looked at the "current" code from my existing receiver but I see some differences there with NTP (which I'm not using actually), and some other things.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 25, 2021, 07:35:31 PM
Hi MTCAT
I think the attached file would be the equivalent under Tinycore. You might have to change  aterm  to  xterm.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 26, 2021, 11:17:20 AM
Hi Rich,

Thanks for the modified script file, I modified the "hard-coded" path's to the config files in the C-code, re-compiled, and changed the path in the script to "/home/tc/.local/bin/..." and I think it worked except now I'm getting a "pipes" communication error, can't open pipes. I did notice a "#define" statement like #define INPIPE "/tmp/mtin" and also #define OUTPIPE "/tmp/mtout" inside "mt_adc.c"

I do have a /tmp directory but,..., still didn't work.

But I could swear that I read in the code that I'm playing around with that Peter abandoned "pipe" communication for the shared memory segment "thing", so I'm a bit confused (as usual).

So, hmm, not sure, perhaps I should just "bite the bullet" and try to get the most current version of the code going, or at least what I think is the most current version, which is the version that's running on my two existing receivers, I guess I have to do that at some point anyway....what do you think ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 27, 2021, 09:32:50 AM
Hi Rich,

One mystery solved, I had two .tgz files in the same backup directory as a bunch of uncompressed code, I assumed that the uncompressed code came out of the .tgz files, I was wrong, the code in the .tgz files (which I expanded on the VortexDX3) is different than the uncompressed code in the backup directory, the source code in the .tgz files must be older, still using pipe communication, while the uncompressed code uses the shared memory technique. When I originally studied the source code I did so with the uncompressed code (hence the abandoning pipes for shared memory comment), so that was my mistake there.

I then also have a third version of the acquisition code which is directly out of the run directory of my existing working receiver. I guess I should just try to move forward with that.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 27, 2021, 11:25:13 AM
Hi MTCAT
You should try to figure out which one is the current working version. Presumably the latest one? Maybe see if there
are revision levels or version numbers listed somewhere. You can see which  .tgz  file is newer by checking the timestamps
of the contents:
Code: [Select]
tar tvf FILENAME.tgzChecking the timestamps of the  .tgz  files won't work because those get changed every time you copy them unless you
use the  -p  switch when copying.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 27, 2021, 06:26:33 PM
Hi Rich,

Mark it on the calendar !, June 27/2021,  with all your help, I got the GUI for the acquisition code to come up ! ! !, :) , this is awesome !, thanks so much for all your help, and Juanito, curaga, bmarkus, patrikg and PDP-8 too (hope I didn't forget anyone)!!!

I used what I think is the newest version, compiled that, and voila ! There is something weird though with Peter's Makefile, I don't think the "make clean" command is working properly to remove the old install, even though I compiled and installed the new version, after running the startup script I initially was still seeing the "pipe error" as if the start script was running the old (and identically named) compiled version, so I just removed the old install manually from ~/.local/bin, and then re did "make" and "make install" and off we go.

Only thing left to do now (I think) is the good old GPS, Peter uses NTP to save a "peerstats" file and a "clockstats" file (in /etc/ntp/stats I think) and then reads a bunch of information out of that, so I just have to figure out how to get chrony to do that to keep Peter's program happy.

Oh, also, Peter has a "sensors" command as well, it almost looks like a DOS batch file command or something in the C code, something like "cmd * = sensors...." or something like that, he uses that (with tail) to get the CPU and board temperature. So need to get that as well.

Because those items are missing (hopefully that's the only reason), the GUI is hung up, none of the tabs or buttons work, but I was really happy to see all the graphics come up anyway..

So I think that might be it, hopefully that won't be too difficult to get going, but otherwise, the program seemed to interrogate the A-D properly, and load all the config files, etc., so just a few small details to go I think.

Very exciting, thanks again for all the help.

The new version of the program also appears to have the ability to do statistics on the PPS, by doing a 12 hr recording of the PPS, and then generates the "CDF" tables as well, so that could be handy. But it's just stated in a comment line that you have to use the "- pps switch" to do the statistics collection, but there's no documentation other than that.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 27, 2021, 08:46:56 PM
Hi MTCAT
... Oh, also, Peter has a "sensors" command as well, it almost looks like a DOS batch file command or something in the C code, something like "cmd * = sensors...." or something like that, he uses that (with tail) to get the CPU and board temperature. So need to get that as well. ...
I've already made the syntax, spelling, punctuation, capitalization speech several times. I can't work with vagaries.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 28, 2021, 12:46:55 PM
Hi Rich,

See line 72 in the attached file, I was working from (scattered) memory after looking through about 50 sub-programs on Sunday, sorry that I didn't get the command exactly right, I was just "talking out loud" so to speak, sorry for the confusion.

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 28, 2021, 12:52:29 PM
Hi Rich,

And excited to have got the GUI to come up, just wanted to tell you about it I guess.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on June 28, 2021, 01:06:41 PM
Hi MTCAT
That program requires you to install  lm_sensors.tcz.
Be sure to read the  info  file:
http://repo.tinycorelinux.net/3.x/tcz/lm_sensors.tcz.info
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on June 28, 2021, 02:40:50 PM
Hi Rich,

Thanks a lot for all the help, you've been far beyond amazing, I'll give that package a try later this week.

Thanks again,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 03, 2021, 12:46:08 PM
Hi Rich,

I installed lm_sensors.tcz and then ran "sudo sensors-detect", unfortunately I didn't detect any sensors at all.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 03, 2021, 12:47:43 PM
Hi Rich,

I also looked in /sys/class/thermal and this directory is empty ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 03, 2021, 01:11:24 PM
Hi MTCAT
So there was no file called  /etc/sysconfig/lm_sensors ?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 03, 2021, 01:39:31 PM
Hi Rich,

That's correct, there was no file "lm_sensors" in /etc/sysconfig, no sensors were detected so I guess it writes out nothing at all ? I did verify that on my old non-RT kernel, I am able to cat /sys/class/thermal/thermal_zone0/temp, but it always returns 66000 (66C), not sure it's really working, and cat /sys/class/thermal/thermal_zone0/type returns acpitz. Hopefully that helps or gives you a clue.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 03, 2021, 01:56:02 PM
Hi MTCAT
... I did verify that on my old non-RT kernel, I am able to cat /sys/class/thermal/thermal_zone0/temp, but it always returns 66000 (66C), not sure it's really working ...
Was that after running  sudo sensors-detect  and loading any kernel modules listed in  /etc/sysconfig/lm_sensors ?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 03, 2021, 02:02:01 PM
Hi Rich,

No, on the original non-RT build, there are three symbolic links inside /sys/class/thermal, one of them being thermal_zone0, there's also a thermal_device0 (I think) and a thermal_device1, but, these have always just been there on the non-RT build, we didn't have to do anything special to get them in there (that I can recall).

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 03, 2021, 02:11:47 PM
Hi MTCAT
I've lost track. Are you running the pre-built RT kernel found here:
http://repo.tinycorelinux.net/3.x/contrib/rt-kernel/

If so, are the dependencies for lm_sensors loaded, including:
hwmon-2.6.33.3-l1-rt19.tcz
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 03, 2021, 02:25:00 PM
Hi Rich,

That's correct, I installed lm_sensors.tcz on the pre-built RT kernel. I installed lm_sensors.tcz from the app tool, so I hope all the dependencies are installed ? I'm not sure how to check that, but I did try to install hwmon-2.6.33.3-l1-rt19.tcz (after already installing lm_sensors.tcz) and I get returned, "hwmon-2.6.33.3-l1-rt19.tcz already installed !" So at least that is there, but the only thing added to onboot.lst is lm_sensors.tcz

Thanks,

David

Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 03, 2021, 02:32:19 PM
Hi MTCAT
In th at case it is loaded. Run:
Code: [Select]
sudo sensors-detect
dmesg > dmesg.txt

Attach  dmesg.txt  to your next post. I'll be back in a couple of hours.

Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 03, 2021, 02:58:52 PM
Hi Rich,

Thanks a lot for the help, here's the dmesg.txt file. The only thing that "sudo sensors-detect" seems to do is load module i2c-dev, which is shown in the last line of the dmesg file I think, after running "sudo sensors-detect", lsmod shows two new entries, "i2c_dev" and an "i2c_core". I also attached the output of "sudo sensors-detect" here.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 03, 2021, 06:29:14 PM
Hi MTCAT
I'm not seeing anything helpful.

... I did verify that on my old non-RT kernel, I am able to cat /sys/class/thermal/thermal_zone0/temp ...
Were there any extra kernel modules loaded there that don't show up under the RT kernel?
Were you using any extra boot codes there that you are not using under the RT kernel?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 04, 2021, 02:56:13 PM
Hi Rich,

I attached the output of lsmod for the "original" non RT kernel here, and dmesg as well.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 04, 2021, 03:08:40 PM
Hi Rich,

Here's the output of lsmod and dmesg for the "home-made" or "built from scratch" RT kernel (this is the RT kernel where we had to run "sudo udevadm trigger" to get the OS to see the ethernet port).

Both the "original" non-RT kernel, and the "home-made" RT kernel are able to see an "ACPI" thermal zone0 which always reports 66 C.

Both the "original" non-RT kernel, and the "home-made" RT kernel have slightly different  ACPI behaviour (see lines 288 to 292 in their respective dmesg files) as compared to the "pre-built" RT kernel.

For some reason, the "pre-built" RT kernel doesn't show this ACPI behaviour, the "thermal zone" entries are missing from it's dmesg file (see latest_dmesg.txt), something must be different with how ACPI is configured perhaps ?

In any case, I'm  a bit suspicious of the 66 C reading when it is available anyway, I don't see any temperature readings at all in the DX3 BIOS, so....., I've queried WinSystems to see if this board (PPM-C412) has any onboard temperature sensors at all, hopefully they'll respond in the next few days.

I suppose if it doesn't have any temperature sensors, I could either add my own somehow ?, or simply comment out the "sensors" line in the acquisition code and feed the program dummy temperature readings, which would be a shame, as that's useful information to know, but would keep the program happy at least.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 04, 2021, 09:13:41 PM
Hi MTCAT
The working kernels don't appear to be loading any modules related to temperature monitoring. It's possible some modules
built into the kernel are being used, but they wouldn't show up using lsmod.

You can see if any of these boot codes make any difference:
Code: [Select]
apm=on
apm=smp
acpi_pm_good
A list of boot codes valid for a 2.6.33 kernel can be found here if you're interested:
http://redsymbol.net/linux-kernel-boot-parameters/2.6.33/

Just a thought, but if temperature really is important, a better option would be to connect a sensor to one of your
A/D channels and get it that way.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 05, 2021, 12:50:37 PM
Hi Rich,

Thanks for the ideas, I tried those boot codes but nothing worked, except sometimes the boot "dead-time" (the dead time when the DX3 sits there seemingly doing nothing after loading the 24DSI driver) seemed to be eliminated, or much reduced.

But, I'm not sure if the DX3 even has a built in temperature sensor, so until I hear otherwise from WinSystems, I may as well just return dummy values in Peter's subroutine for temperature, I did find out that lm_sensors won't work with the acpitz "digital" sensor, see the "Hardware Support" part, laptop discussion, in here

https://github.com/lm-sensors/lm-sensors

For now, I think I'll move on to getting chrony to save the necessary log files (position, date, time, etc) and hopefully get the acquisition code up and running at least.

Inside the attached program, lines 61 to 90, I think I could comment out all the lines except for lines 61, 90, and then change lines 87 and 88 to something like,
*cpu_temp=50.0;
*board_temp=50.0;

I think that would work ?, to simply return dummy values of 50.0 C from subroutine get_temperatures ?

Thanks,

David

Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 05, 2021, 12:54:02 PM
Hi Rich,

Sorry, forgot line 68 would need to remain uncommented as well, so all lines commented except for 61,90,68 and then change lines 87, 88 to dummy values of 50.0 C.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 05, 2021, 01:29:23 PM

Hi MTCAT
Paste this before the  get_temperatures  routine:
Code: [Select]
static int get_temperatures(double *cpu_temp, double *board_temp)
{
    /* Returns dummy temperature.
     */

    *cpu_temp = 50.0;
    *board_temp = 50.0;

    return (0);
}
Rename the original routine to  get_temperatures_old. That should make it clear in the future if the code is examined
that there was/is an issue here.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 07, 2021, 04:59:53 AM
Thanks Rich, I'll do that, that's a great idea to make a whole new subroutine, thanks again. I did find some little USB-stick temperature sensors that I could use (I think) if I want to down the road, could just attach the thermocouple to the CPU heat sink to get a ~ CPU temperature, better than nothing.

https://www.dracal.com/store/products/usbtenki/index.php

https://www.tindie.com/products/electromake/usb-thermometer-ds18b20-win-linux-pc-rpi-2/

I've got chrony saving log files now, so that's good, may have to put a "rm /home/tc/chrony/logs/statistics.log" or similar in a startup script though else the file size will get out of hand eventually, the log files just continue to grow for as long chrony runs, even across power up/power down cycles. I put the log files in home so the directory structure would be persistent but downside is I will need to wipe them out on every boot-up I think. I've had it up and running for maybe an hour or so, and the two log files (statistics.log, tracking.log) are about 20 kB each (if I recall correctly), not massive, but,could pile up eventually. Then just have to figure out Peter's little command he uses to pull out the clock offset, etc., from these files.

Then, last thing (hopefully) will be to make a little script "sudo cat /dev/ttyS0 > nmea.txt" and get Peter's program to pull the location (latitude/longitude) out of the .txt file, the "trick" will be, how long do I have to wait in general before I have valid GPS position, not sure, maybe I could just put the above command into the start_logger script so that as we start the logger, we also pump out some NMEA sentences ? When the device doesn't move more than a few hundred km, the GPS starts up quite fast (30 seconds or so ?), but when moved to a new position, it takes longer (several mintues, maybe even 5 minutes) as it has to search for the new satellite constellation first (I think).

This is fun doing all these tweaks to get the program to work, feels like it's near the end, then just have to re-do the whole process on 4 or 5 more, yikes !

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 07, 2021, 06:56:56 AM
Hi Rich,

While gpsd is running (collecting all the GPS info off ttyS0) I guess I can't run "sudo cat /dev/ttyS0" w/o messing up gpsd and therefore also chrony. Looks like I need to use a program called "gpspipe", I notice that this in the TC 3.x repository under "gpsd-clients.tcz", hopefully I won't mess anything up by already having "gpsd.tcz" installed....or, I could run "sudo cat /dev/ttyS0 > nmea.txt" before starting gpsd and chrony, that could work too...., as long as the GPS was "happy" at that point.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 07, 2021, 07:33:54 AM
Hi MTCAT
... I did find some little USB-stick temperature sensors that I could use ...
The first thing I would do is run this command (Yes, the backslash in front of the underscore is intentional):
Code: [Select]
grep -r "get\_temperatures" Path/To/Sourcecode/*Then use the result to see what is actually being done with the temperature(s).

Quote
...  the log files just continue to grow for as long chrony runs, ...
The attached file shows a command to trim a text file to the last 100 (or whatever number you choose) lines.


Quote
... how long do I have to wait in general before I have valid GPS position, ...
This should tell you if chrony is synchronized to a satellite:
Code: [Select]
chrony sources | grep "\*"
The description for this is:
Quote
S
    This column indicates the state of the sources. "*" indicates the source to which chronyd is currently synchronized. "+" indicates acceptable sources which are combined with the selected source. "-" indicates acceptable sources which are excluded by the combining algorithm. "?" indicates sources to which connectivity has been lost or whose packets do not pass all tests. "x" indicates a clock which chronyd thinks is a falseticker ( its time is inconsistent with a majority of other sources). "~" indicates a source whose time appears to have too much variability. The "?" condition is also shown at start-up, until at least 3 samples have been gathered from it.

This should tell you how far off your clock is:
Code: [Select]
chrony tracking | grep "System time"
System time     : 0.000001501 seconds slow of NTP time

The description for this is:
Quote
System time
    In normal operation, chronyd never steps the system clock, because any jump in the timescale can have adverse consequences for certain application programs. Instead, any error in the system clock is corrected by slightly speeding up or slowing down the system clock until the error has been removed, and then returning to the system clock's normal speed. A consequence of this is that there will be a period when the system clock (as read by other programs using the gettimeofday() system call, or by the date command in the shell) will be different from chronyd's estimate of the current true time (which it reports to NTP clients when it is operating in server mode). The value reported on this line is the difference due to this effect.

Found here:
https://jfearn.fedorapeople.org/fdocs/en-US/Fedora_Draft_Documentation/0.1/html/System_Administrators_Guide/sect-Checking_if_chrony_is_synchronized.html
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 08, 2021, 05:11:35 AM
Hi Rich,

Thanks so much for all that info, and that little script, that could be very useful! I'll have lots of things to try this weekend, hopefully I can get the acquisition program fully up and running this weekend.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 11, 2021, 03:37:00 PM
Hi Rich,

I made the new subroutine to return dummy temperature values (and re-named and commented out the original one). In the new subroutine, I don't need to declare "cpu_temp" and "board_temp" as double variables since that's done right in the subroutine definition line ?, right at the start ?, just wanted to make sure, that's definitely different than Fortran !, would find out in any case when compiling I suppose !

I'm trying to get the offset and error out of the chrony log file located in /home/tc/chrony/log/statistics.log, I thought I would try it this way first since this is how Peter does it (with ntp log files though).

If I execute "tail -2 /home/tc/chrony/log/statistics.log" we'll get what's shown in out.txt file attached here. I want the numbers in the second and third columns (of numbers), that's the offset and the error (std. deviation) respectively (I think I would want just the second row though, which is for the PPS source).

Executing "tail -1 /home/tc/chrony/log/statistics.log | cut -d ' ' -f 1" gives me the date, like "2021-07-11", so I think that's good.

And then executing "tail -1 /home/tc/chrony/log/statistics.log | cut -d ' ' -f 2" gives me the time, like "20:37:56", so I think that's good as well.

But, executing "tail -1 /home/tc/chrony/log/statistics.log | cut -d ' ' -f 18" works for the second column and executing "tail -1 /home/tc/chrony/log/statistics.log | cut -d ' ' -f 20" works for the third column, but only when all the numbers are positive, a negative sign in there will mess things up since I'm using the ' ' (a space) as the delimiter.

Is there a good way to get around that with a better use of the "cut" command ?

Otherwise, I would need to get the offset and error from the output of "chronyc sources" (like you suggested) but I think even there I would have the same issue when running something like, "chronyc sources grep PPS | cut -d ' ' -f 41" (for the offset) and then "chronyc sources grep PPS | cut -d ' ' -f 46" (for the error), but again, this isn't foolproof since the output of "chronyc sources" isn't always "fixed format".

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 11, 2021, 03:49:15 PM
Hi Rich,

Sorry, I should have referred to the variables as "*cpu_temp" and "*board_temp", not used to seeing that asterisk there, I guess this indicates a pointer to a memory location ? (from what I've been able to read), quite different than Fortran...!

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 11, 2021, 07:46:01 PM
Hi MTCAT
... (and re-named and commented out the original one). ...
No need to comment it out. By renaming it, the compiler won't generate any code for it because it's not referenced
anywhere else in the program.

Quote
... I don't need to declare "cpu_temp" and "board_temp" as double variables ...
Correct. The subroutine is being passed pointers to doubles. When manipulating pointers, this alters the pointer:
Code: [Select]
cpu_temp=This alters what the pointer is pointing to:
Code: [Select]
*cpu_temp=
Quote
... I'm trying to get the offset and error out of the chrony log ...
Don't do multiple tail commands to parse the same line. The file could get updated between 2 tail commands.
I've attached a file that shows what you want to do.
Line 1 assigns the result of the tail command to a variable.
Line 2 echos the contents of the variable. You'll note the whitespace has been reduced to 1 space between each field.
Lines 4 and 5 show the echo and cut command assigning the values you are looking for to  offset  and  error.
Line 6 echos the contents of  offset  and  error.


*** Pay close attention to backtics  `  vs apostrophes (single quotes)  '. ***
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 13, 2021, 05:04:32 PM
Hi Rich,

Thanks for the explanation regarding the pointers, that's something new I've now learned, thanks.

Thanks for the commands to do a nice job of parsing the chrony output file, I'll try that this weekend.

Then I just need to get the NMEA sentences from the GPS, either before gpsd starts up and "hogs" ttyS0, or with "gpspipe" (to communicate properly with gpsd). I've already installed the gpsd package (gpsd.tcz) from the TC3.x repository but in order to get "gpspipe" it looks like I need to install "gpsd-clients.tcz", hopefully that will be okay to install gpsd-clients.tcz with gpsd.tcz already installed.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 13, 2021, 08:45:27 PM
Hi MTCAT
Thanks for the explanation regarding the pointers, that's something new I've now learned, thanks. ...
Fair warning, there's more to pointers than what I described.
This declares a pointer to a double:
Code: [Select]
double *cpu_temp;
This declares 2 doubles, then a pointer to a double and initializes the pointer to celcius. What it ls pointing
to (celcius) is unaffected:
Code: [Select]
double board_temp;
double celcius;
double *cpu_temp=&celcius;

get_temperatures expects 2 pointers to doubles. The first variable is a pointer. The second variable is
the address of a double, which is the same thing a pointer would pass to the subroutine.
Code: [Select]
get_temperatures(cpu_temp, &board_temp);
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 16, 2021, 01:56:55 PM
Hi Rich,

Referring to the attached program, line 117, would it work if I replaced line 117 with a "call" to a script file, lets say it's called "parse_chrony.sh" containing the commands in parse.txt you attached previously (except maybe for lines 2 and 6, the echo to screen commands).

char *cmd="/home/tc/parse_chrony.sh"

As I understand it, popen puts the output of the command in the string into a file called "in" in my case (due to using the "r" option), and then down below (line 122), Peter uses fgets to put the contents of  the file "in" into a character variable called "buf" I think, and then uses "sscanf" (line 131) to read the offset and the jitter as doubles out of "buf" (I don't why the !=2 is in the sscanf line though, something not equal to 2), and then outputting the offset and jitter in milli-seconds (if it's less than 0.25 seconds dt=dt*1000), or, outputting 1000.0 (milli-seconds) if the offset or jitter is > 0.25.

It would seem much easier to just use your nice commands to save the offset and error values into a file, and then just load that file, and return those values ?

Thanks,

David


I think that's what's going on there in subroutine get_pps_offset ?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 16, 2021, 06:20:08 PM
Hi MTCAT
Referring to the attached program, line 117, would it work if I replaced line 117 with a "call" to a script file, lets say it's called "parse_chrony.sh" containing the commands in parse.txt you attached previously (except maybe for lines 2 and 6, the echo to screen commands).

char *cmd="/home/tc/parse_chrony.sh" ...
Yes, that should work, but you want the second echo command to echo the 2 variables being read from  peerstats
by  get_pps_offset. You should not need to change anything else to make it work.

Quote
... uses "sscanf" (line 131) to read the offset and the jitter as doubles out of "buf" ...
And that's precisely what should be in  buf , no more, no less.

Quote
... I don't why the !=2 is in the sscanf line though ...
sscanf  returns the number of items successfully matched and assigned. If 2 items are not processed, then -1 is
returned indicating an error.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 19, 2021, 10:54:22 AM
Hi Rich,

Oh, okay, thanks, I'll keep that second echo command in there then, unfortunately I didn't get a chance to do anything this past weekend, hopefully will try this tomorrow or Wednesday at latest. Then I just have to get gpspipe going to retrieve NMEA sentences, I think "gpspipe -d -o nmea.txt" would work to output NMEA sentences into a text file "nmea.txt", and then just need another little script file to read out the lat-long and number of satellites out of "nmea.txt", and hopefully that will be it !, hopefully Peter's program will be 100 percent happy at that point !

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 20, 2021, 06:32:04 PM
Hi Rich,

I put your nice commands into a script file called "parse_chrony.sh" and looks great, thanks a lot. I downloaded gspd-clients.tcz to get the "gpspipe" utility which is able to retrieve NMEA sentences from gpsd, I ran  "gpspipe -r -n 100 > nmea.txt" and attached the file here. Would you happen to know of a good way (in terms of bash commands) to extract only the $GNGGA sentence(s?) in the attached text file ? If you can show me how to get only the $GNGGA sentences into a new file (which I'm sure you can !), maybe called "gngga.txt" ?, Peter's command (line 195 in gps_ublox.c) should do the trick I think, in my case it would be "tail -1 /home/tc/chrony/log/gngga.txt | cut -d ',' -f 3,4,5,6,7,8"

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 20, 2021, 06:56:52 PM
Hi MTCAT
The following will create a new  gngga.txt  file every time you run it:
Code: [Select]
grep '$GNGGA' nmea.txt > /home/tc/chrony/log/gngga.txt
This will create a new  gngga.txt  file the first time it is run, and append to the existing  gngga.txt  thereafter:
Code: [Select]
grep '$GNGGA' nmea.txt >> /home/tc/chrony/log/gngga.txt
Note the single quotes used so that  grep  looks for a literal dollar sign.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 21, 2021, 06:51:10 AM
Hi Rich,

You're amazing, thanks a lot Rich.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 21, 2021, 10:02:50 AM
Hi Rich,

I got script files setup to extract the offset and error from /home/tc/chrony/log/statistics.log (parse_chrony.sh), and another one to extract the GNGGA sentence from /home/tc/chrony/log/nmea.txt (parse_nmea.sh), I then modified "gps_ublox.c" to call these script files to get the required information, okay, cool. Unfortunately when compiling though, I'm getting an error related to the new "get_temperatures" subroutine which "dummies" out the temperature to 50.0, I attached the output from stderror here, and the modified gps_ublox.c routine as well. Did I do the comments wrong on the original get_temperatures subroutine ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 21, 2021, 10:24:35 AM
Hi MTCAT
This marks the beginning of a comment  /*
This marks the end of a comment  */
You don't need additional asterisks on each line.
You also can not comment out a comment, i.e  /* some C code /* A comment */ */

I'm going to make the same recommendation I made previously, don't comment out the original code. Just rename
the original routine, which you did. Remove your attempt at commenting it out.

Quote
gps_ublox.c: In function 'get_temperatures':
gps_ublox.c:67: error: invalid operands to binary * (have 'double' and 'double *')
gps_ublox.c:69: error: expected ';' before 'return'
gps_ublox.c:70: warning: no return statement in function returning non-void
You omitted the semicolons that mark the end of a statement. Look at what I wrote again:
http://forum.tinycorelinux.net/index.php/topic,24944.msg160104.html#msg160104
Then compare that to what you wrote.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 21, 2021, 10:33:26 AM
Hi Rich,

Thanks for the quick reply, I'll undo my commenting out attempt of the "get_temperatures_original" subroutine and add those semi-colons in the "get_temperatures" subroutine, sorry I missed that.

Thanks for the info re: commenting in C, interesting.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 21, 2021, 12:02:46 PM
Hi Rich,

Thanks again, that worked, mt_logger fully compiled now with no problems. But, when I tried to start the acquisition program (./start_logger_TC.sh), the GUI is still frozen, the normal acquisition GUI screen comes up, but then all of the buttons and tabs within the acquisition window are still non-responsive (frozen), not sure where to start on that in terms of troubleshooting. When I minimize the acquisition window, and then bring it back up, it comes back just as a blank off-white screen with "MT Logger" in the upper left corner, maybe that's a hint ?, maybe an issue with this glade library or something ?

Thanks,

David

Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 21, 2021, 12:11:21 PM
Hi Rich,

Sorry for all the posts, I did actually find an error in "mt_adc.log", see line 60 attached here, some kind of issue with the A-D driver it seems ? I'll contact GSC as well regarding this error, the GSC test programs have always run okay, so I'm not sure why we would be having an issue now.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 21, 2021, 01:01:16 PM
Hi MTCAT
Sorry, I can't tell you anything based on that log.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 21, 2021, 01:39:11 PM
Hi Rich,

I re-ran the "./rxrate", "./id" and "./billion" test programs for the 3.17.52 driver and they all work fine, but, I do notice that the GSC test programs just report "(DMA)" for their communication, if that's the right word. But, for some reason, in Peter's program, it's "DMDMA" (Demand-Mode DMA) that's being attempted to be used. I'm tempted to uncomment the line in adc_control.c that's specifies "regular" DMA and then comment out the "DMDMA" line immediately below it, just to see if that works.

But, bigger picture, that makes me concerned again that I'm still not working from the same source-code that produced the working executables on my two functioning receivers, maybe there's no way to truly tell the correct version other than trial and error ?

Still waiting to hear back from GSC.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 21, 2021, 02:39:17 PM
Hi Rich,

No difference when using the other DMA mode (adc_control.c), I did find in the main program (mt_adc.c), attached here, that there is some extra debugging info, lines 146/147 were commented before, I uncommented them and re-ran and got a bit of extra information shown in the attached log file, not sure how this is supposed to help though.

I also re-tried running the program with lower sample rates of 100,000 samples/sec and even 10,000 samples/sec, still got the same error.

Still waiting to hear from GSC.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 21, 2021, 06:42:27 PM
Hi Rich,

I found out that the "error" I'm seeing in "mt_adc.log" is actually to be expected, it seems to be a consequence of shutting down mt_adc when clicking on the 'x' on the terminal window in which it's running. I actually see this same "error" in the log file on my properly functioning receiver, sigh. I think mt_adc runs continuously and when it's abruptly turned off by closing the terminal, complains a bit.

I did find one problem, maybe, Peter has the data directory defined in "setup.c" (for mt_logger) simply "home/data" (yet it's reported in the mt_logger,conf file as /home/emp/data, which is for my Slackware boxes), I forgot to change this to "/home/tc/data" in setup.c, I did this, but to no effect, and in fact, it doesn't seem that the mt_logger.conf file is being written out at all, maybe because the program is not fully up and running yet.

So I don't know, maybe it is a graphics (glade ?) issue, the GUI is basically frozen, although I did get it to switch over to the "Channels" tab one time, not sure how that happened, but the GUI did respond kind of, once anyway. There's a strange script file (attached here), looks pretty odd to me, maybe glade is not getting setup properly somehow for the new display situation (VDX3, 15 inch VGA monitor, as opposed to the old situation, which is a small 7 inch (?) LCD display), I could just hook up one of the LCD screens to see if it's still using the "old" settings or something ?

But, I'll be gone til the end of next week, so, will have to wait I guess.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 21, 2021, 06:44:50 PM
Hi Rich,

Sorry, Peter defined data directory was "/home/data" in setup.c, changed to "'home/tc/data".

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on July 21, 2021, 06:46:12 PM
Wow, must be bedtime, "/home/data" changed to "/home/tc/data", sorry.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 21, 2021, 06:48:40 PM
Hi MTCAT
All I can do is make some observations:
Quote
block_index start_read  end_read
 146 21:19:11.845  21:19:11.860
 145 21:19:11.743  21:19:11.845
 144 21:19:11.640  21:19:11.743
 -----Snip-----
The difference between start and end times is 102 to 103 milliseconds for blocks 137 to 145.
For block 146 the difference dropped to 15 milliseconds.

Quote
mt_adc halted - exit code = -2
The -2 exit code is the error code returned by:
Code: [Select]
read_status = read_adc(&ring_buffer[block_index * BLKSIZE], bufsize, error_msg);
Quote
ERROR: read() failure, errno = 110
That's a timeout error. Probably occurring in  read_adc().
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on July 21, 2021, 06:59:59 PM
Hi MTCAT
... There's a strange script file (attached here), looks pretty odd to me, maybe glade is not getting setup properly ...
The script appears to create a header file containing program version and compilation date for display by the GUI.

This comment suggests the script gets run by a make file when the program gets compiled:
Quote
ui.h - created by makefile from *.glade file
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 08, 2021, 02:58:52 PM
Hi Rich,

I've been reading up on GLADE and GTK+ and have unfortunately not really gotten anywhere. I tried today to re-compile the MT_logger program with safer compile options, previously I was compiling with all sorts of Pentium specific compiler options, Peter had CFLAGS= -Wall -march=pentium3 -O3 -ffast-math -mmmx -msse -malign-double $(shell pkg-config --cflags gtk+2.0), but he also had a "safer" CFLAGS line defined as "CFLAGS= -Wall -g -march=native -O2 -malign-double -mfpmath=sse $(shell pkg-config --cflags gtk+2.0).

But unfortunately, even when compiling with the "safe" compiler options I still ended up with a hung up GUI....

I did try typing "pkg-config --cflags gtk+2.0" in the MT_logger build directory and got the following error;
Package gtk+2.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `gtk+2.0.pc'
to the PKG_CONFIG_PATH environment variable
No package gtk+2.0 found

Maybe this is the reason for the problem ??, but strange that I don't get any errors when running "./make" ?

Thanks,

David
 
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 08, 2021, 03:06:25 PM
Hi Rich,

Sorry, forgot the ending quotes on the "safe" compile option line, should be "CFLAGS= -Wall -g -march=native -O2 -malign-double -mfpmath=sse $(shell pkg-config --cflags gtk+2.0)".

Also, the command to compile the MT_logger code is just "make" (typed in the build directory, /home/tc/newest_mt_loggerv1), this is also the directory where I tried to manually execute the pkg-config command. First I type "make clean" (removes old compiled version from /home/tc/.local/bin) and then "make" to compile, followed by "make install" (to copy executable to /home/tc/.local/bin).

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 08, 2021, 03:31:04 PM
Hi Rich,

I just looked in /mnt/sdb1/tce/onboot.lst, I've installed "gtk2-dev.tcz", maybe I should have installed "gtk2.tcz" ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on August 08, 2021, 05:45:09 PM
Hi MTCAT
... Package gtk+2.0 was not found in the pkg-config search path. ...
Try changing  gtk+2.0  to  gtk+-2.0.

I just looked in /mnt/sdb1/tce/onboot.lst, I've installed "gtk2-dev.tcz", maybe I should have installed "gtk2.tcz" ?
No.  When you load  gtk2-dev.tcz  it loads  gtk2.tcz  automatically because its a dependency of  gtk2-dev.tcz.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 10, 2021, 05:34:28 AM
Hi Rich,

I'm sorry, inside the Makefile the command already is gtk+-2.0, I mistyped it when trying to run it manually, I need new glasses I guess, running the proper command at the terminal seems to work fine...darn-it, I don't know what else to try except maybe put the equivalent of "print *, a", etc., throughout the 40 or so individual C++ programs to try to see where the program is hanging up ? Or maybe try gdb ?, the "safe" compile CLFAGS line has the "-g" option which is used to generate debugging info for gdb I think ?

Thanks,

David
 
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on August 10, 2021, 06:50:15 AM
Hi MTCAT
... Maybe this is the reason for the problem ??, but strange that I don't get any errors when running "./make" ?
Another typo?
That  ./  would tell the operating system the  make  command is in the current directory, it is not. It's in  /usr/local/bin/.
The  make  command looks in the current directory for a file called  makefile  or  Makefile  in that order.

You could try  make -d  on the off chance the debugging information  make  prints out is useful.

Did you execute  MT_logger  by itself from the command line to see if it prints any warnings/errors?
Does the previously compiled version of  MT_logger  that you had run?

What happens if you replace  -march=native  with  -march=i486 -mtune=i686  in the  "safe"  version?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 12, 2021, 03:43:02 AM
Hi Rich,

I replaced -march=native with -march=i486 -mtune=i686 and I then got a bunch of warnings saying SSE instructions were being disabled for 387 arithmetics, but when running this version, still got a hung up GUI. Then I removed the -mfpmath=sse line as well and recompiled, that got rid of the SSE warnings, but still no luck with the GUI.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 12, 2021, 04:08:53 AM
Hi Rich,

I also tried typing out the commands in start_logger_TC.sh and I didn't see any errors come back, but the program behaviour was different, when running the script, three terminal windows get opened, and they stay open, but when I typed the commands out manually, terminal windows just popped up briefly and then disappeared. I guess I'll try gdb next...? I also tried running all commands in the start_logger_TC.sh as root by adding a sudo to the second and third commands in the script, but that didn't help, the first command in the start_logger_TC.sh script has a sudo in there, but that didn't help either.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 12, 2021, 04:47:24 AM
Hi Rich,

Last thing i tried this morning was to comment out the last line in start_logger_TC.sh so that when I execute the script (./start_logger_TC.sh) it only starts mt_adc and opens the adc log file. Then, in another window, I typed "gdb mt_logger", followed by "break main", and then typed "next" over and over until something happened.
Inside "main.c" there's a function (subroutine ?) called gtk_init(  ), the code hangs up inside gtk_init. As I type "next" repeatedly inside gdb, I saw a bunch of commands coming out of "lib/libc.s06" and then as I kept going, I started to see commands coming out of "lib/ld-linux.s0.2 and eventually saw a segmentation fault as below.

0xb77a0371 in _dl_debug_state () from lib/ld-linux.s0.2
(gdb) next
Single stepping until exit from function _dl_debug_state,
which has no line number information.

Program received signal SIGSEGV, Segmentation Fault.
0x00000003 in ?? ()

Also, in gdb, at the top of the window, it indicates "multi-thre Thread 0xb69ea In:   Line: ??  PC: 0x3

Could there be an issue of some sort with the dual core Vortex as well maybe ? The original application was a on a single core Pentium....

Hopefully there's something in there that jumps out at you, I also attached main.c here.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 12, 2021, 04:54:55 AM
Hi Rich,

By the way, I'm not sure, I wonder if one of the inputs (argv) to gtk_init is junk (not being assigned), this is what I see at the very start of running gdb,

Starting program: /home/tc/newest_mt_loggerv1/mt_logger
[Thread debugging using libthread_db enabled]
Breakpoint 1, main (argc=1, argv=0xbf9c7d14) at main.c:19
(gdb)

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on August 12, 2021, 10:24:03 AM
Hi MTCAT
... Breakpoint 1, main (argc=1, argv=0xbf9c7d14) at main.c:19
argc tells you the total number of arguments that make up that command.
argv is an array of strings, each one containing one of the arguments that make up that command.
In this instance, argv contains one string (argv[0]) that if printed would display:
Code: [Select]
/home/tc/newest_mt_loggerv1/mt_logger
... I typed "gdb mt_logger", followed by "break main", and then typed "next" over and over ...
Code: [Select]
gdb mt_logger
run
When it segfaults, you can get a stack trace like this:
Code: [Select]
bt
You said  start_logger_TC.sh  starts 3 programs. Are you sure the other 2 are working? What happens if either of
those 2 are not functioning correctly?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 12, 2021, 06:39:19 PM
Hi Rich,

The startup script starts two programs, mt_adc and mt_logger, mt_adc starts first in a terminal window, and then a second command uses tail to show the output of the mt_adc log file in a second (separate) terminal window, and then lastly mt_logger starts in a third terminal window, and creates/opens up the GUI.

But you're right, I have no idea whether the problem is with mt_adc, or mt_logger, I just assumed it was mt_logger since that's where all the GUI stuff is and my first symptom of a problem was a hung GUI.

I'm not sure, but I think mt_adc is basically sampling all the time and then the mt_logger program is responsible for "catching" transient events and displaying them in the GUI, and it's the mt_logger program where you can change sample rate, A-D scale, etc, inside the GUI.

When I run mt_adc in gdb it just runs, I can see it with "ps" as well, I just get a solid green cursor in the terminal window that's running gdb. I'm not totally sure, but I think it's doing what it's supposed too.

Back to my "original" testing method, which is to run the start script with the first two commands only, and then run mt_logger in gdb in another window, I'm seeing the segmentation fault, but now in a different spot ??? Tonight it (mt_logger) seems to get farther into the code before the segmentation fault happens ??? By the way, when I just type "gdb mt_logger" and "run", gdb gets hung up (I think) and I have to hit Crtl-C to get the prompt back, I can backtrace then but not sure that's valid ?

I did my creep through the mt_logger code with "next" and now I see the following,"Program received signal SIGSEGV, Segmentation fault. No function contains program counter for selected frame." And then when I type "bt" I see,
#0 0x00000003 in ?? ()
#1 0x00000005 in ?? ()
#2 0x0xbfb498cc in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack ?)

I don't know why in gdb when I type "layout next" I'm seeing assembly language code now too, sorry, can't figure anything out here...

I did try just running "gdb mt_logger", and then "run" and then "Ctrl-C" when it hangs, and then "bt", this is what I get in that case,"#13 0x080512a7 in main (argc=1, argv=0xbfcc4d64) at main.c:34"

This is different as this is quite near the last line of main.c now ?? ( gtk_main( ) )

Sorry for this novel, thrashing I guess, confused myself even further (if that's possible)...

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on August 12, 2021, 10:05:02 PM
Hi MTCAT
... mt_adc starts first in a terminal window, and then a second command uses tail to show the output of the mt_adc log file in a second (separate) terminal window, ...
OK, when you do that, does the second window show data being produced? Should it show data being produced?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 13, 2021, 04:32:28 AM
Hi Rich,

Thanks for the help, yeah, the second terminal window shows the results of the 24DSI12 startup and intitialization tests and the log file (mt_adc.log) is also "fresh" judging by the time stamp on the file (it saves a new one on every startup), it is odd though, the log file (in /home/tc/logs) shows as an executable file I think, not related to the problem most likely but seems odd ?

I've also noticed another issue (surprise, surprise), if I start mt_adc once, and then try to start again, I have problems with shared memory, I get an error that the shared memory segment (if that's the right word) already exists, I have to do a full re-boot every time it seems.

I'm not sure that's the case with my fully functioning receivers, although I usually do just turn it on, collect data at a station, and then turn it off and get to the next station, so could be the same thing there.

Thanks,

David
 
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on August 13, 2021, 06:25:20 AM
Hi MTCAT
... shows the results of the 24DSI12 startup and intitialization tests and the log file (mt_adc.log) is also "fresh" judging by the time stamp on the file  ...
But is the 24DSI12 collecting and saving data?

Quote
... it is odd though, the log file (in /home/tc/logs) shows as an executable file I think ...
Assuming its not the program doing it, that will happen if a file contacts a file system that does not support permissions,
such as FAT. Running  ls -l /home/tc/logs  will show all the bits set (-rwxrwxrwx).

Quote
... I'm not sure that's the case with my fully functioning receivers ...
You might want to check that. It may or may not provide an additional clue as to what's going on.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 13, 2021, 07:13:55 AM
Hi Rich,

No, there's no digitized data being saved to disk right now (I don't think anyway, I should look on the CF card to make sure), I can't tell for sure if the 24DSI12 is collecting anything at all, both of those functions are seen within the GUI (mt_logger), which is frozen of course. I think mt_logger is responsible for grabbing the output of mt_adc, filtering and decimating it to create a "continuous" data stream (@ 20 kHz sample rate), and then triggering and localizing "transient" events at the full 200 kHz sample rate, and saving those transient files too.

I'm not totally sure, but I think mt_adc is working but there's something hanging up in mt_logger, maybe something to do with threads ? Grasping at straws here, Peter is using "-lpthread" link flag in the Makefile for mt_adc, yet the only programs that have an include statement "#include <pthread.h>" are part of the mt_logger build ???, rc_server.c and another one I can't recall off the top of my head. I found a post yesterday about using "gthread" and changing a link flag to "gthread-2.0" to solve a seg fault in gtk_init, I tried to compile mt_adc with a link option "-lpthread-2.0" but that didn't work, gcc complained about that...

I guess it would be ideal if I knew the overall "flow-chart" of the program, and could step through it to see where the problem is, that's what I was trying to do with gdb but I'm getting kind of confusing results out of gbd, last night I wasn't seeing the seg fault occurring in gtk_init now, so where does the seg fault actually occur ??

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on August 13, 2021, 08:09:46 AM
Hi MTCAT
... I found a post yesterday about using "gthread" and changing a link flag to "gthread-2.0" to solve a seg fault in gtk_init, I tried to compile mt_adc with a link option "-lpthread-2.0" but that didn't work, gcc complained about that...
How did you get from  gthread-2.0  to  -lpthread-2.0?

I suspect there's more to it than just changing a link flag:
I don't know if  #include <pthread.h>  would need to be removed.
You probably need to add  #include <glib.h>  if it's not already there.
You might have to change the syntax of any thread statements.
The link flag would be  -lgthread-2.0

I've never programmed using threads so I don't know any of that for certain.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 13, 2021, 08:27:33 AM
Hi Rich,

Just an act of pure desperation !, I thought there might be a "2.0" option for pthread as well, but it didn't work, and as you say, it would be too easy if it was just a link flag.

I'm a bit overwhelmed with mt_logger especially, the final executable being the result of linking 33 separate object files !!, to me, it would be way easier if it was written sequentially in one big file, at least then I would feel like I better know the structure of the program, the way it is right now, to me anyway, it just seems all over the place, what programs are executed in what order ???? I think it would help to debug if I knew the proper path through the code so to speak, I am aware of "profiling" but I think I need a proper functioning executable for that to work ?

Maybe the only choice is to continue with gdb, but I'm not sure why the segfault seemed to occur in a different spot last night.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 13, 2021, 05:42:50 PM
Hi Rich,

Home from work and fired up gdb, I'm not seeing segfaults unless I step through the program with "next", when I just do the startup script to open two terminal windows, one running mt_adc and another showing the output of mt_adc.log, and start mt_logger manually in another terminal by typing "gdb mt_logger" and then "run", I get a hung GUI, I hit Ctrl-C in the gdb window, and then "backtrace", the last command mt_logger gets to is line 218 in filter.c, which is just applying a 4'th order Butterworth low-pass filter (used for low-pass filtering before downsampling to make the 20 kHz data stream I think).

Attached filter.c here, in gdb, when I type "print tic" I get 154666080, and for "print i" I get 1, I also can see f_counter (from backtrace) as 0x9380414 which converts to 154666004, but yet when I type "print f_counter" I get returned "<value optimized out>".

The "tic" value is not correct, it should be between 0 and 4, I don't even know how order=4 is getting passed in to filter.c, I could try passing in order=2 (if I could find the right spot in the 33 separate C programs) to see if that solves the problem by avoiding apply_bwf_filter4 and apply the second order filter instead.

I'm not sure if you can see any issues with apply_bwf_filter4 ? It's odd too, in the backtrace, it shows "order=4" as an argument of apply_bwf_filter4 right after "nchan" but if you look at 184 in filter.c, there's no "order" in that list of arguments that I can see ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on August 13, 2021, 08:03:01 PM
Hi MTCAT
The compiler optimizations are screwing up the gdb output. Set the optimization flag to  -O0 (that's oh zero).
Recompile and see if the gdb output looks any better.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Juanito on August 14, 2021, 12:34:50 AM
You can also substitute "-ggdb" for "-march=i486 -mtune=i686 -Os -pipe" - in addition, you may need to search the Makefile for "-g" and "-O2" and remove them.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 15, 2021, 08:27:23 AM
Hi Rich (and Juanito),

I changed the optimizer setting to "-O0" and was able to get somewhat better information out of gdb. I did also find where the low-pass filter order is set (to make the downsampled continuous data).

I then re-ran the code with both second order and fourth order filters (with the "-O0" optimization).

In both cases, the code still gets stuck in filter.c, in the filter loop, either "apply_bwf_filter4" or "apply_bwf_filter2".

In both cases, I seem to be getting nonsensical values for data input to the filter (and of course for data output from the filter).

In both cases, I'm seeing input data (when converted to decimal) are like 2.8 x 10^9 (integer), max integer value for a 24 bit ADC is ~ 16,777,216 .

It doesn't always seem to bomb on the same iteration, but "i" is usually 0 or 1, in the case of the 4'th order filter routine "tic" was 1, another time it was 2, so at least they are reasonable numbers.

Seems like garbage input right now ?, which would mean of course that I'm not getting proper data from the ADC, which would mean mt_adc is not working properly still.

Not sure how to test that, should be at least seeing bit-noise of the A-D getting through, so, need to test mt_adc somehow I think.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on August 15, 2021, 08:37:56 AM
Hi MTCAT
I don't know how much difference it will make, but I would also add  -ggdb.  I just checked some of my compile
scripts, and I use a  GDEBUG  variable that when set compiles with  -O0 -ggdb  for running with  gdb.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 16, 2021, 05:09:46 PM
Hi Rich,

I tried the -ggdb compile flag as well and it didn't really add anything (that I could tell anyway). I'm going to try to get a consultant to login to the little computer remotely and see if he can debug the code, I just don't know C at all (although slowly learning with your help) but shudder to think how long it may take to get the code running at the current pace.

I'm attempting to install "Team Viewer" (version 8, the oldest one, and 32 bit version) for the remote login but I'm missing the following libraries; libasound.so.2 and libwine.so.1.

Need to install WINE I guess ?, not sure where libasound.so.2 would be found ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 16, 2021, 05:45:24 PM
Hi Rich,

Found both libraries in the good old repository ! Software seems to work !, wow !, so apparently now this fellow should be able to log in to my little computer and get the C code working !, hopefully anyway.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 17, 2021, 05:40:07 PM
Hi Rich,

What would be the best way to make a clone of my existing TinyCore real-time pen drive (sdb1, 16 GBytes). I tried using Clonezilla to do the job but it didn't work for some reason...I'm a little gun-shy to use "dd" but maybe that would be best ? Would the below be okay ? The bigger sdc1 drive (64 GByte) was originally a FAT32 (W95) volume, Clonezilla changed that, but something went wrong as when I plug in the big 64GByte drive in TinyCore now, it shows up in dmesg and fdisk -l okay (as sdc1) but it doesn't show up in the mount tool.

dd if=/dev/sdb1 of=/dev/sdc1

Obviously don't want to mess up the 8 months of progress to date, but thought it might be good to have a spare duplicate pen drive.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on August 17, 2021, 08:12:59 PM
Hi MTCAT
Plug in both drives, but do not mount them. Assuming you want to copy  sdb  to  sdc , try this:
Code: [Select]
dd if=/dev/sdb of=/dev/sdc conv=noerror,sync,fsyncThen go do something else. This will take a while.

noerror instructs dd to continue operation, ignoring all read errors. Default behavior for dd is to halt at any error.
sync fills input blocks with zeroes if there were any read errors, so data offsets stay in sync.
fsync instructs dd to physically write data out before finishing.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 18, 2021, 04:42:30 AM
Thanks Rich, that worked great ! I was confused about mounting (or not) the drives too, thanks for explaining that.

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on August 18, 2021, 08:27:16 AM
Hi MTCAT
You are welcome. Thank you for confirming it worked.

Just a couple of notes:
1. If someone comes along and says  "hey, that  dd  command will run a lot faster if you add  bs=64k" , don't do it.
   dd  defaults to  bs=512 , so it reads and then writes 512 bytes at a time. With the  noerror,sync  flags, if you get
   a read error, it will zero out that block. While a larger block size (i.e. bs=64k) will run faster, that same read error
   would zero out a much larger (64k) block.

2. Run  sync  and wait for the command prompt to return before removing your devices. I'll add that to my last post.

3. Your  64 GByte  device should look like your  16 GByte  device, with 48 GBytes unallocated. Use a program like
   gparted  to expand your partition or add partitions if you want to be able to use those 48 GBytes.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 23, 2021, 07:15:05 AM
Hi Rich,

That's good to know, I was tempted to try using a larger block-size, but I didn't, I think it took about 45 minutes to do the job with the default blocksize, so not too bad really. I didn't do the "sync" part though, but I did verify that the newly created pen-drive boots up the system just fine, so that was great, this will be very useful in the future when we get everything working properly, can just clone a bunch more pen drives !

On other matters, I got ssh installed and a fellow from Brazil (freelancer.com) is logging in remotely to see if he can figure out why the acquisition program is hanging up, I have a feeling it's something simple for someone who knows what they're doing with C code...I guess I'm missing out on a learning opportunity in a sense but I need to get this going to move on to other things, hopefully this guy will be able to find the problem..

I did have a strange occurrence this weekend though, not sure what I did to cause this, maybe the monitor wasn't turned on during boot up ?, but when I booted up the little Vortex I ended getting stuck at a "XVesa Resolution" screen, with choices of "0-1 or q(uit)", not sure how that happened, I just gave it the three finger salute and started over and has been okay ever since then, I'm trying to recall the error from memory but it was something like a missing "xession" file or something, I actually have a post written out at home and can send you the exact error message if you want.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on August 23, 2021, 08:38:31 AM
Hi MTCAT
... I didn't do the "sync" part though, but I did verify that the newly created pen-drive boots up the system just fine ...
If you lost data it would have been near the end of the device, so it would still boot up. I presume the 16 GByte device
was not full, so if you pulled it early you likely didn't lose any data.

For the future, you can do the command like this:
Code: [Select]
dd if=/dev/sdb of=/dev/sdc conv=noerror,sync,fsyncThat should force dd to sync the device before returning the command prompt. I'll update my other post to reflect that.

... I actually have a post written out at home and can send you the exact error message if you want.
Attach it as a  .txt  file in your next post.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 23, 2021, 04:18:14 PM
Hi Rich,

Here's the .txt file describing what happened a couple of mornings ago.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on August 23, 2021, 04:21:08 PM
Hi Rich,

Forgot to add, after I hit "q", I did end up at a command prompt, and then I hit Ctrl-Alt-Del to re-boot. So technically TC did boot, but couldn't start flwm for some reason.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on August 23, 2021, 05:24:21 PM
Hi MTCAT
I'm not sure why that happened.
It appears  xsetup.sh  was called because  startx  could not find  /home/tc/.xsession.
Because you quit  xsetup.sh , it did not create  /home/tc/.xsession.
Control then returned to  startx  and it tried to execute  /home/tc/.xsession  which still did not exist.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on September 03, 2021, 06:11:59 PM
Hi Rich,

The acquisition program is mostly working !, all I needed to do was input a PPS signal to the ADC ! ! A remote programmer stepped through the C program to find where the problem was, and I immediately clued in as to what was going on, very embarassing. As you know, the acquisition code does both "burst" recording of time localized transient events, but also collects a down-sampled continuous data-stream. The continuous data is time synchronized so that each one minute file (1,200,000 samples) starts and stops at the beginning and end of a one minute time period, but it needs the PPS input to CH6 of the 24DSI12 in order to do it, that's all it really was ! The GUI was frozen as it didn't have the PPS there on CH6 to synchronize the continuous data-stream ! !

There is one oddity though, the symbolic link I made (with your direction of course) points home/tc/data to the CF card (128 Gbyte), but Peter's program indicates that home/tc/data only has 2 GBytes free space left ! So there's something odd with my Lubuntu install on the CF card, the program that copied the image file of Lubuntu onto the CF-card (can't recall the name) did something undesirable so that there's a lot of wasted space on the CF-card it seems.

So I could just re-format the CF card and get rid of Lubuntu altogether ?, or, maybe I could even install TinyCore onto the CF-card and just work everything off that.

Otherwise, I just need to get this USB port temperature sensor going so that I have some meaningful temperatures to report too..by the way, those neat commands you made up for me to parse the chrony files are working like a charm, the lat/long position and clock offset information is being displayed in the program perfectly, thanks a lot.

Finally, almost there, thanks again for everything, now I just have to build 5 more or so ! Would be great if the program could be made to run on both cores of the Vortex too, it's a little bit sluggish at the moment (running on one core), but I'll still enjoy the 12W or so power saving and the lighter battery needed to run it all day !

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on September 03, 2021, 06:32:38 PM
Hi MTCAT
... There is one oddity though, the symbolic link I made (with your direction of course) points home/tc/data to the CF card (128 Gbyte), but Peter's program indicates that home/tc/data only has 2 GBytes free space left ! ...

And that is why I mentioned this after you ran the  dd  command:
...
3. Your  64 GByte  device should look like your  16 GByte  device, with 48 GBytes unallocated. Use a program like
   gparted  to expand your partition or add partitions if you want to be able to use those 48 GBytes.

Or are you referring to a different CF card?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on September 03, 2021, 06:51:51 PM
Hi MTCAT
I just reread the statement. Install  gparted.tcz. Then from the command line:
Code: [Select]
sudo gparted /dev/hdaReplace  hda  with the designation for the CF card. This should give you a picture of how all of the space is allocated
on your device.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 03, 2021, 12:22:06 PM
Hi Rich,

Sorry for taking so long to get back to you, I was away for a few weeks....I installed gparted.tcz and then executed "sudo gparted /dev/sda5" and then a nice GUI came up reporting sda5 as 6 GByte's ? I then clicked on "View --> Device Information" and saw the following;

Model: Unknown
Size: 6.00 GiB
Path: /dev/sda5
Partition Table: loop
Heads: 255
Sectors/track: 63
Cylinders: 783
Total sectors: 12587008
Sector Size: 512

I also re-directed the output of "fdisk -l" into a text file attached here, also, in FLWM, in the mount tool or app, I only see "sda5" and "sdb1" as mountable partitions, of course sdb1 is my boot pen drive (16 GBytes).

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 03, 2021, 12:54:46 PM
Hi Rich,

Executing "sudo gparted /dev/sda" worked a bit better, read your post again, sorry, showed me that 112.25 GBytes are unallocated....,/dev/sda1 is a "linux-swap" of size 1019 MBytes, /dev/sda2 is "extended" of size 6 GBytes, and it appears that /dev/sda5 is a sub-category under /dev/sda2 and is of type "ext4" of size 6 GBytes, 4.10 GBytes of which is used, and 112.25 GBytes are unallocated ?

Maybe I need to re-format the CF-card and in the process "nuke" Lubuntu ? I think that would be okay, the only thing Lubuntu has going for it really is that it's able to see the 1000 MBit/sec ethernet port on the DX3, maybe TinyCore 3.8.4 could be made to see that too but at the moment TC only "sees" the 100 MBit/sec ethernet port, which is not too bad anyway.

On my existing (Pentium) receivers I use the ethernet with "scp" command to copy data off the receiver (daily), can be 15 GBytes or more sometimes, with 100 MBit/sec ethernet port (on my old Pentium SBC) I'm getting around 6 MBytes/sec copy speed, takes as long as 1 hour just to b/u the data to an external SSD, would be nice if I could use the faster ethernet port on the DX3, or maybe even the USB 2.0 ports on the Vortex DX3 would be faster than 6 MBytes/sec ? Believe it or not, the old Pentium SBC has only two USB 1.0 ports !

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 03, 2021, 04:17:23 PM
Hi MTCAT
First:
Code: [Select]
sudo gparted /dev/sda
Right click the  sda5  line.
Click  Resize/Move  in the popup menu.
Select your new size.
Click the  Resize  button.
Click the green checkmark in the toolbar and wait for the operation to complete.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 05, 2021, 05:24:07 PM
Hi Rich,

Thanks for the help, unfortunately I'm not able to resize sda5 though, after running "sudo gparted /dev/sda" I had to unmount sda5 before "resize/move" became active, but even after unmounting, right clicking on sda5, and selecting "resize/move", it shows sda5 as minimum/maximum size of 6146 MBytes ?, I can click on the unallocated space entry though, and it looks like I could create a new partition of size 112.25 GBytes but then I would also need to change the symbolic link (/home/tc/data ---> /mnt/sda5/data) and the little script in /opt that mounts sda5.

Or, should I unmount sda5, delete it, and then absorb that ~ 6 GBytes into the 112.25 GBytes and then make a new sda5 partition that's 112.25+6 GBytes in size ?

Thanks,

David

Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 05, 2021, 05:57:03 PM
Hi MTCAT
OK, I see the problem.

First expand the  Extended  partition (sda2) to the maximum size (about 118 GBytes or so).
Then expand sda5 to whatever size you want.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 09, 2021, 01:48:41 PM
Hi Rich,

Sorry to be such a pain, I was able to resize sda2 to ~ 118 GBytes but sda5 is still "locked in" at just over 6 GBytes (6146 MBytes), I can't seem to change that still, minimum size, 6146 MBytes, maximum size, 6146 MBytes, only the space preceding and following on sda5 is adjustable right now with gparted, size of sda5 seems fixed at 6146 MBytes ?, sorry for the hassle.

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 09, 2021, 05:03:57 PM
Hi MTCAT
If you reboot, does it work then?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 09, 2021, 05:53:09 PM
Hi MTCAT
... I was able to resize sda2 to ~ 118 GBytes but sda5 is still "locked in" at just over 6 GBytes ...
After you to told it to resize, did you remember to click the green checkmark in the toolbar? The resize operation won't
take place until you click the green checkmark.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 12, 2021, 05:56:07 PM
Hi Rich,

Yeah, I did click the green checkmark to apply the operation (I noticed that were "pending" operations otherwise), so it did indeed resize sda2, sda2 is now a little over 118 GBytes but sda5 is still just 6 GBytes?, I attached the output of "fdisk -l" here, showing sda2 nice and big, but sda5 still the same size....

I guess if worst comes to worst, I could just reformat the entire CF card ? Would be getting rid of Lubuntu, would only possibly be using it for the 1000 MBit ethernet anyway, maybe could get that working in TinyCore too.

Thanks,

David

Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 12, 2021, 06:15:25 PM
Hi MTCAT
So with  sda5  not mounted, you can't change the value of the  New size  field in the  Resize  popup?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 13, 2021, 05:27:51 PM
Hi Rich, with sda5 not mounted, I right click on sda5, then click on resize/move, and then a new window pops up with four entries, top one is "Free space preceding (MiB)", next one below is "New Size (MiB)" and next one below is "Free Space following (MiB)" and bottom one is "Align to". Only problem is, the "New Size" entry is greyed out, so I can't change it, only the "free space preceding" and "free space following" and "align to" entries are changeable, so no matter what I do here sda5 seems fixed at 6146 MB, also, in the top of the window (after right clicking on sda5 and then clicking on resize/move), in that window, in the top it reports "Minimum size 6146 (Mib)" and Maximum size 6146 (MiB)".

If I cloned my boot pen drive onto the CF-card (using dd like you taught me) and then used gparted to expand the partition to the full 128 GBytes, I guess I could boot off the CF-card then and just work that way ? Of course losing Lubuntu, and some of the scripts in /opt would need to change (to auto mount sda5, and also the symbolic link for home/tc/data -> /mnt/sda5/data would need to be removed.)

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 13, 2021, 07:02:58 PM
Hi MTCAT
See if running a file system repair on the unmounted device helps:
Code: [Select]
sudo e2fsck -fp /dev/sda1
sudo e2fsck -fp /dev/sda2
sudo e2fsck -fp /dev/sda5
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 14, 2021, 06:04:45 PM
Hi Rich,

I unmounted sda5 and then tried the "sudo e2fsck -fp /dev/sda1" command and I got a warning message that sda1 is mounted, it gave me the option to quit, so I did.

Then I tried the command on sda2 I got an interesting message back, "Attempt to read block from filesystem resulted in a short read while trying to open /dev/sda2, could this be a zero length partition ?"

And then trying the command on sda5, I got;
"/dev/sda5: 196094/393984 files (0.1 % non-contiguous), 1078723/1572864 blocks"

I then tried "sudo gparted /dev/sda" again but still see the same behaviour, min/max size of sda5 6146 MB.

Weird thing too, I simply typed "mount" to see if sda1 shows up but I couldn't see it anywhere, all I saw was "/dev/sdb1" and a bunch of "/dev/loopxxx" where "xxx" runs from 0 to 122.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 14, 2021, 08:38:50 PM
Hi MTCAT
I unmounted sda5 and then tried the "sudo e2fsck -fp /dev/sda1" command and I got a warning message that sda1 is mounted, it gave me the option to quit, so I did. ...
When I said  "unmounted device"  that meant all partitions on the device.

Quote
... Then I tried the command on sda2 ...
Forget about sda2. Turns out you can't run  e2fsck  on an extended partition.

Quote
... I simply typed "mount" to see if sda1 shows up but I couldn't see it anywhere ...
Try this to remove the clutter:
Code: [Select]
mount | grep -v loop
Or just run:
Code: [Select]
sudo umount /dev/sda1
Then run:
Code: [Select]
sudo e2fsck -fp /dev/sda1
Code: [Select]
sudo e2fsck -fp /dev/sda5
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 15, 2021, 05:03:01 AM
Hi Rich,

Sorry for all this hassle, I still can't unmount sda1, typing "sudo umount /dev/sda1" I get an error "can't unmount sda1: Invalid argument", I also tried in gparted, by right clicking on sda1, and then selecting "Swap off" (sda1 shows up in gparted with a file-system of type "linux-swap") and gparted wouldn't let me do that either, I get an error "Could not deactivate swap", under the flags column in gparted, it also says "boot" for sda1, so I guess sda1 is the boot sector for Lubuntu ?, the "mount | grep -v loop" didn't show me an sda1 but sdb1 was there of course.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 15, 2021, 05:28:43 AM
Hi MTCAT
Boot with  sda  not plugged in so Lubuntu can not use the swap partition on it. Then plug it in. Then run  fdisk -l  to
confirm what designation (sda, sdb, etc.) your device was assigned.  Then make sure Lubuntu did not auto mount
the partitions on it.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 15, 2021, 03:41:25 PM
Hi Rich,

Thanks for the help, I did as instructed but when I run "fdisk -l" all I see is the pen drive, which is now labeled "sda1". I tried two different timings with respect to re-insertion of the CF-card but got the same result, once I didn't re-insert the CF-card until TC was fully booted and FLWM was running, and the second time, I re-inserted the CF-card at the point of the 10 second time out during boot, but same result either way, can only see the pen drive with "fdisk -l".

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 15, 2021, 05:00:56 PM
Hi MTCAT
... can only see the pen drive with "fdisk -l".
If you can only see the partitions with  fdisk , that means they are not mounted. That's what you want.
Yes, your device may have been assigned a different designation, that's OK.
Run the  e2fsck  command on partition 1 then 5.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 15, 2021, 06:02:55 PM
Hi Rich,

Sorry for the confusion, I didn't explain well enough, the only partition I can see when running "fdisk -l" is the pen drive, which gets designated sda1 when the CF card is not plugged in at boot time. I can't see the CF card at all unless it's plugged in at power up it seems ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 15, 2021, 07:00:33 PM
Hi MTCAT
OK, so you are booting Tinycore from the pen drive, and the CF card contains the partition you are trying to expand.

Leave the CF card plugged in.
Boot from the pen drive but add the boot code:
Code: [Select]
noswapThis way it won't use partition #1.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 16, 2021, 08:46:32 AM
Hi Rich,

Oh, but I thought that the linux-swap partition was just for Lubuntu, tinycore doesn't use it anyway......I'm really okay with nuking Lubuntu, would you be able to help me with getting everything (Tinycore and data acq program) going off just the CF-card, I think that's the way I would want to go anyway, I could just use dd to clone the existing boot pen drive (16 GBytes) onto the CF-card, and then (hopefully) use gparted to expand the partition to use the whole 128 GBytes on the CF-card.

I guess there would be a bit of mucking around with a couple scripts in /opt (one which auto mounts sda5, and another which makes a symbolic link), but I think that would be it, not too bad I don't think.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 16, 2021, 04:18:05 PM
Hi MTCAT
Oh, but I thought that the linux-swap partition was just for Lubuntu, tinycore doesn't use it anyway...
If Tinycore finds a swap partition, it will use it.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 17, 2021, 02:47:36 PM
Hi Rich,

Oh, okay, I'll try the "noswap" boot code and then run gparted and see what happens.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 17, 2021, 03:16:47 PM
Hi Rich,

I added the "noswap" boot code to extlinux.conf and I saw during bootup that it indeed was not using any swap, then I unmounted sda5 and tried running b"e2fsck -fp /dev/sda1", I got an error, "e2fsck: Bad magic number in super-block while trying to open /dev/sda1 :", and then the rest of the message I attached here in a text file.

I also ran "e2fsck -fp /dev/sda5"   and got the same result as before, and lastly, I ran "sudo gparted /dev/sda" and once again I still couldn't resize sda5, it's still stuck at 6146 MBytes....

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 17, 2021, 05:45:56 PM
Hi MTCAT
Quote
... If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt ...
Since  sda1  is a swap partition, the attached error message is saying it's not a problem.

Your last fdisk attachment contained this:
Quote
Partition 1 does not end on cylinder boundary
It also looks like sda1 overlaps sda2 & sda5. Lets try fixing that and see if that helps.

Code: [Select]
sudo gparted /dev/sdaSelect resize for sda1
Shrink it by about 5 megabytes
Click the  Resize  button
Click the green  Apply  check mark

Once it finishes resizing sda1, see if it lets you resize sda5.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 17, 2021, 06:08:04 PM
Hi MTCAT
I just thought of one more thing:

In gparted, click  View->File System Support
Make sure the  Grow  and  Shrink  columns have green check marks for  ext2, ext3, ext4, and linux-swap.
If they don't, the  Required Software  column will tell you what you need to install.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 18, 2021, 05:49:15 PM
Hi Rich, that might be it !, I'm missing "e2fsprogs" (for ext2 and ext3) and "e2fsprogs v1.41+" (for ext4), I'll try and get that !

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 18, 2021, 06:12:22 PM
Hi MTCAT
In TC3, for  ext2, ext3, and ext4 , you want  e2fsprogs_apps.tcz.
For  linux-swap , you want  util-linux-ng.tcz.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 18, 2021, 07:21:28 PM
Hi Rich, thanks !, that did it !, I had all green check marks for linux-swap already but installing the e2fsprogs_apps.tcz package was needed for ext2, ext3 and ext4. I was able to re-size sda5 to almost 119 GBytes now, so that's pretty good, but..., even after I installed the e2fsprogs_apps.tcz package I still couldn't resize sda1, I got a message, "an error occurred during the requested operation" and no other information than that, so out of frustration, I thought, well, what can I do with sda1, can I delete it ?, turns out, yes I can, so sorry, I hope that doesn't come back to haunt me, but after getting rid of sda1, and installing the package, I was able to re-size sda5 to almost 119 GBytes. Thanks for all the help, now I need to try to compile the code with -O2 or -O3 and see if I can speed it up, it's a bit of a dog right now, oh, and get that USB temperature sensor going too.

Thanks again for all your help, and your patience.

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 18, 2021, 07:28:41 PM
Hi MTCAT
You can recreate the swap partition if you left that section of the device unallocated.
Right click on the unallocated area and select  New
Select  linux-swap  for  File system
Click  Add
Click  Apply
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 24, 2021, 03:25:35 PM
Hi Rich,

Thanks, at least there's an option, I ordered a few more new CF-cards today, I'll keep my current one as is (with Lubuntu) but on the new ones I think I'm going to try just using dd to copy over the existing boot pen drive contents onto the CF-card, and see if I can boot from the CF-card, and then obviously save data right to the CF-card too, so just run off the CF-card, and that's it, would be a bit simpler hopefully.

Right now the acquisition program works (or did), but is quite slow (in the GUI-changing tabs takes a second or two-seems boggy), so I removed the "-ggdb" flag in the Makefile and also changed "-O0" to "-O3" (go for the gusto) and now make is having problems finding a package "xcb" required by X11 ?, and also can't the header file for gtk ? I don't know what I did as this was all compiling just fine before, sigh.

Attached a screenshot here, I'm sorry but I couldn't re-direct the output of make into a .txt file for some reason, or all it showed was "main.c".

I also attached onboot.lst where you can see that libx11-xcb.tcz, libxcb.tcz,  and gtk2-dev.tcz are all being loaded, so I'm confused, as usual as to what make is complaining about, sorry for this never-ending line of questions from me.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 24, 2021, 03:33:00 PM
Hi RIch,

I forgot to add, I changed the Makefile back to its original configuration with "-ggdb" and "-O0" and still got the same error, I attached the Makefile and main.c here as well...sorry.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 24, 2021, 06:02:32 PM
Hi MTCAT
You have  libxcb.tcz  listed in your  onboot.lst  file. Maybe that should have been  libxcb-dev.tcz ?

Does running this eliminate the error:
Code: [Select]
tce-load -i libxcb-dev
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Juanito on October 24, 2021, 10:16:08 PM
You can use pkg-config to check for missing dependencies and recursive dependencies, like this:
Code: [Select]
$ pkg-config --cflags xcb
Package xcb was not found in the pkg-config search path.
Perhaps you should add the directory containing `xcb.pc'

Then you can use the apps gui to find which extension provides xcb.pc and load it:
Code: [Select]
$ tce-load -i libxcb-dev
$ pkg-config --cflags xcb
-I/usr/local/include
$ pkg-config --libs xcb
-L/usr/local/lib -lxcb

This is particularly useful for apps with a number of recursive dependencies like gtk+-2.0 and gtk+-3.0 as it will show which of the dependencies is missing.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 25, 2021, 04:51:05 PM
Hi Rich and Juanito,

Thanks for the help, I'll give that a try, before I do though, I was wondering, could installing gparted.tcz and e2sfprogs_apps.tcz have messed up something ? Maybe the pkg_config_path got messed up somehow ? I was able to compile the acquisition program no problem before installing those packages, so, what changed ?, the only changes that were made was installing gparted and e2sfprogs, and deleting sda1 (linux-swap) ?, and now I get this error when trying to compile, maybe just a coincidence ?

Thanks,

David

Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 25, 2021, 05:04:03 PM
Hi Rich and Juanito,

Sorry for all the posts, I tried "echo $PKG_CONFIG_PATH" and it returned an empty string, nothing at all, is that correct ?

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 25, 2021, 05:19:20 PM
Hi MTCAT
I doubt installing  gparted.tcz  and  e2sfprogs_apps.tcz  caused it. If you want to test that, open your  onboot.lst  file
and delete the  gparted.tcz  and  e2sfprogs_apps.tcz  lines. Then reboot and try compiling again.

I suspect  libxcb-dev.tcz  needs to be loaded.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 25, 2021, 05:45:45 PM
Hi Rich,

I'm just wondering because the acquisition program compiled just fine before with just libxcb.tcz ?, with onboot.lst as attached previously but w/o gparted.tcz and e2fsprogs_apps.tcz

I think make couldn't find xcb package but it also couldn't find the gtk header file ?

Does it seem correct that "echo $PKG_CONFIG_PATH" would return an empty line ?, that's what I'm getting right now, I assume that if installing gparted.tcz and/or e2fsprogs_apps.tcz messed up PKG_CONFIG_PATH that I would maybe just need to "re-set" the PKG_CONFIG_PATH, but I'm not sure what it should be.

If I'm barking up the wrong tree, I'll stop now, just seems odd that the program compiled okay with just libxcb.tcz before installing gparted and e2fsprogs_apps.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 25, 2021, 06:02:42 PM
Hi MTCAT
I would not normally expect  PKG_CONFIG_PATH  to be set in a normal environment.

Did you try Juanitos suggestion in reply #359 ?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 25, 2021, 06:24:06 PM
Hi Rich,

Oh, okay, no I haven't tried anything yet, I think Juanito's suggestion is really just the same as yours ?, right ?, namely, install libxcb-dev.tcz, but then Juanito also showed how to export the compiler and library flags, is that right ?

Thanks,

David

Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 25, 2021, 06:35:35 PM
Hi MTCAT
... but then Juanito also showed how to export the compiler and library flags, is that right ?
Yes. That's how a script or make file retrieves the flags a particular  -dev  package requires.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 25, 2021, 07:04:19 PM
Hi Rich and Juanito,

Sorry for being stubborn about libxcb-dev.tcz, you guys were right, I installed it and mt_logger compiled no problem, don't know why it compiled before okay with libxcb.tcz but will remain a mystery I guess !, thanks for the help, now I can see if the "O3" made the GUI any snappier !

Although I notice that the mt_adc Makefile has no optimization flag set in it all, and still uses the "-g" option. Lots of things to play around with I guess.

Thanks again, you guys saved the day for the 100'th time once again !

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 25, 2021, 07:19:43 PM
Hi MTCAT
... don't know why it compiled before okay with libxcb.tcz but will remain a mystery I guess ! ...
Either  libxcb-dev.tcz  was getting loaded before or you were compiling non-GUI parts of the program.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 28, 2021, 05:09:00 PM
Hi Rich,

I've been playing around with the acquisition program and have noticed a few things. The "-O3" optimization for mt_logger has definitely
perked things up a bit but I'm getting an error which crashes the acquisition program ("adc timeout error") which I think is related to the
volume of data being transferred from the 24DSI into RAM (buffer overflow ?) - I was getting this error with "-O0" compilation as well.

I'm thinking it's due to data volume as I only see it happening with 200 kHz sample rate (which is max), at 100 kHz sample rate per channel, the
program seems pretty happy to run along with no problems at all, for like an hour or more. At 200 kHz sample rate though,
I usually see this error within something on the order of 15 minutes or so.

Peter defined two variables, "blk_size" and "dma_size", as shown in the attached file, lines 59 to 73. This is for setting up the "ring"
buffer and for initiating DMA transfers, right now, DMA transfers occur every 120,000 samples and the ring buffer is about 120 MBytes.

As you might recall, you helped me set up a 256 MByte shared memory segment for the acquisition program, which I thought was just for the ring
buffer alone, but it seems likes it's also used for other things (communication buffer and maybe others). I used 256 MBytes with the VortexDX3
as this is what Peter indicated in the comments of one of the programs, but technically, I haven't seen what is actually being
used on the Slackware boxes, but the original Pentium "only" has 512 MBytes total RAM, so I assume Peter did use a 256 MByte shared memory segment
there too.

But, the VortexDX3 has 2048 MBytes RAM, so we could go a lot bigger, and the 24DSI has a data buffer of 256,000 samples, so Peter is initiating
DMA's when the 24DSI onboard buffer is a little less than half full.

Do you think it might help to use a larger "blk_size" and "dma_size" with the VortexDX3 ?,240 k samples for example- as shown in the comments, and
possibly increase the size of the shared memory segment to 512 MBytes ? (ring buffer would be ~ 240 MBytes - is 256 MByte shared memory segment
large enough in that case ?, could use 512 MBytes just to be safe, kind of have RAM "to burn" with the VortexDX3).

Hopefully that wouldn't just be delaying the inevitable though, i.e., would still have the same failure, but would just take longer to do so.
Based on the General Standards test script "rxrate" it seems that the VortexDX3 can keep up with the 24DSI12, even at max sample rate, throughput
on the bus was good enough, it admittedly only runs for about 10 seconds or so, but maybe the VortexDX3 is being "hassled" by too many DMA
transfers ?, do less DMA's, but larger ones when we do ?

When this "adc timeout error" happens its quite harsh in the sense that it requires a complete re-boot of the computer before the acquisition
program will run again.

Another error I see is more gentle, "adc restart timeout error" which is happening when I try to change sample rate, or ADC scale, etc.,
the acquisition program attempts to "restart" the ADC with the new parameters but it seems to be taking too long and "times-out", there's one
spot where the program is set to give 20 seconds max (I think) to reset the ADC, weird thing is, when I do a change of parameters, I get this
"adc restart timeout error", I then close the acquisition program, then re-start it, and I can see that the desired change did actually take place,
but I'm still getting this error, so maybe it's just barely timing out so to speak. There's also a bunch of "usleep(xxx)" commands sprinkled
throughout, maybe some of those have to be tweaked too.

I'm hoping that if I simply increase the 20 (seconds ?) to say 30 seconds, then maybe it would be okay. The VortexDX3 does take noticeably
longer than the single core Pentium to start up the acquisition program, so it makes sense that I might be "timing-out" relative to the Pentium.


Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 28, 2021, 05:25:51 PM
Hi Rich,

Sorry, noticed a typo, in mt_adc.c the variables are "BLKSIZE" and "DMASIZE", it's in another program (adc_control.c) where "dma_size" is referred too.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 29, 2021, 09:50:25 PM
Hi MTCAT
... at 100 kHz sample rate per channel, the program seems pretty happy to run along with no problems at all, for like an hour or more. At 200 kHz sample rate though, I usually see this error within something on the order of 15 minutes or so. ...
That sounds like the ADC is collecting data faster than it can be transferred to shared memory.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on October 30, 2021, 06:00:44 AM
Hi MTCAT
You could try changing  DMASIZE  to 60K:
Code: [Select]
#define DMASIZE ((60*1024)) /* buffer threshold for dma operations */so the buffer doesn't fill up as much before a DMA transfer starts.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on October 30, 2021, 10:41:02 AM
Hi Rich,

I found that the error is coming from a general standards function (?) called "dsi_dsl_read", so I contacted General Standards and the fellow there sent me a neat little program (hopefully I can get it to work) that monitors the level of the 24DSI12 onboard buffer (percentage of fill I think) and saves this number to disk 1000 times a second. Don from GSC is recommending that I run this program, and then start the acquisition program at the 200 kHz sample rate and let it run until it presumably overflows (10 or 15 minutes), and can then see what's going in with the 24DSI12 onboard buffer.

Sounds like a good idea but, I can see already in his Makefile that I'll need to change the location of the libraries, and he's given some instruction already for changing arguments in a couple place, so hopefully it will actually build.

I guess if I can't it to get work, I'll just try the 60k sample DMASIZE and see how that looks.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on November 04, 2021, 03:39:01 AM
Hi Rich,

As I suspected, Don's buffer examination program is written for the 4.x 24DSI12 driver and won't work with the older driver I'm using, I did run "./rxrate -r10000" to pump 10 GBytes (I guess ?) through the system, it ran fine for almost 20 minutes straight, flat out at 200 kHz, could try a 200 min test I suppose, but otherwise, I did get the full acquisition program to run for a few hours at 200 kHz sample rate with a larger 240ksample dma_blk_size, but results are kind of erratic with respect to the dma_blk_size, 120 ksample is perfect at 100 kHz (it seems), 240ksample at 200 kHz seems pretty good, but still not 100 percent I think.

Seems like there's something else going on as well, the Vortex is having a bit of a hard time with display related duties I think, the continuous stream, which can be displayed as it's collected (if the continuous tab in the GUI is selected), seems kind of "jerky" and I notice the seconds counter in the GUI jumps a bit irregularly sometimes to "catch up" with tinierclock.sh for example, it's not just a steady tick-tock kind of thing within the GUI, there is an integer variable called "loop_time_ms" which is set at 50 (ms) for which the comment says "run the acquisition loop at 20 Hz", I think this is the rate at which the GLADE GUI is refreshed, maybe could try a slower refresh rate of 100 (ms)...I'm also using the VGA output right now, but will use the LVDS output when it's finally done, I don't know if the LVDS output is more "direct" (faster ?).

Otherwise, I was also trying to investigate threads, to see if mt_logger or mt_adc are using multiple threads at all, which Don says is essential (to have separate read and write threads), I tried using "ps -o nlwp <pid>" but the "-o" option doesn't seem to work with the version of ps I have. I also tried looking in /proc/<pid>/task but it didn't make sense to me, there's supposed to be one directory for every thread but there were many directories in there with what looked like system related things, not specific to mt_logger, also "top -H" doesn't work with my version of top (to see threads).

Would you happen to know the best way to see threads for a given process in TC ?

Don also suggested commenting out the one and only include <pthread.h> statement in one of the mt_logger sub-programs and see what errors come up, he thought that this would also tell us where threaded operations, if any, are happening. It's weird though, the one and only include <pthread.h> is inside one of the mt_logger sub-programs but there is no -lpthread compile option in the mt_logger Makefile, but, there is a -lpthread compile option inside the mt_adc Makefile, so seems odd.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on November 04, 2021, 05:00:38 AM
Hi MTCAT
... Seems like there's something else going on as well, the Vortex is having a bit of a hard time with display related duties I think, the continuous stream, which can be displayed as it's collected (if the continuous tab in the GUI is selected), seems kind of "jerky" and I notice the seconds counter in the GUI jumps a bit irregularly sometimes to "catch up" with tinierclock.sh for example, ...
If you disable or at least minimize screen updates, does it still choke at 200 KHz?
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on November 05, 2021, 04:56:53 AM
Hi Rich,

Last night it ran for a little over 3 hrs at 200 kHz with a 240k dma_blk_size leaving the GUI on the transient screen, I collected a few transients but overall there was very little screen refreshing going on, about the only thing that got updated really was the time. I think the dma_blk_size does help as some (smaller) sizes are definitely worse, running for like 10 or 20 minutes, so maybe the larger block sizes helps to mask this "other" issue, which maybe is the screen updating ?

In the end I'll be running this off the LVDS line, on a small 9 (?) inch LCD screen, right now I'm using VGA output on a "big" 15 inch NEC LCD monitor, maybe that's irrelevant, I don't know.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on November 12, 2021, 06:20:22 PM
Hi Rich,

I still haven't been able to get the acquisition program to run 100 % reliably at 200 kHz sample rate, I did find some old e-mails from 2015 from Peter who had the exact same issue with the Pentium SBC we originally purchased, Peter said he solved this problem by changing from pipe communication betweem mt_adc and mt_logger to shared memory. Only problem is, I'm already using the shared memory version of the program with the Vortex so...., do you think it would be worthwhile to "profile" the program to ideally see where the Vortex is spending the most time ? I guess I can compile with "-pg" option, run the program, exit normally, and then run "gprof" on a file called gmon.out.

Otherwise, maybe I just have to accept running at 100 kHz max, it seems 100 percent reliable at that.

I could try running it on a smaller display, at lower resolution, I'm not sure if that would make a difference though, there's a lot of information out there on how to use Cairo graphics library for fast drawing, most people say 20 fps is about typical, which is what Peter is using as well (50 ms), I tried 100 ms (10 fps) and it actually was worse, more jerky of course but also failed even faster, also tried a faster 40 fps (25 ms) and that didn't seem to help either.

By the way, I was able to get rid of that error when re-setting the parameters from within the GUI, so now I can change sample rate, ADC scales, etc, w/o getting an error and having to re-start the program, so that's something at least.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on November 12, 2021, 06:59:54 PM
Hi MTCAT
Video activity can be resource intensive. The way I understand it, DMA transfers take place whenever the system bus is idle.
That means while the CPU is fetching instructions to execute or updating the display, DMA activity will be blocked.

Having said that, you might want to address the question in reply #375.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on November 13, 2021, 05:23:00 PM
Hi Rich,

Sorry, I wasn't dodging that question but was doing more analysis, yes, unfortunately, the program still does choke at 200 kHz sample rate, even when it's left on the transient screen with minimal screen updating, and seems like the dma_blk_size has a minimal effect as well, once it gets larger than a certain size it doesn't seem to matter very much, i.e., 120 k sample seems about as good as 240 k sample ( I think anyway).

The loop_interval_timer, which I think controls how fast the the GUI is refreshed didn't seem to help, I tried both 10 Hz and 40 Hz, the "normal" setting is 20 Hz.

I ran it today at 100 kHz sample rate (240k sample blk_dma_size) leaving it on the continuous screen, and it ran for almost 8 hrs, it crashed not too long after I brought the screen back to life, was also going to try turning off the screen saver, I found a post of yours that says "xset s off -dpms" will do it. So not even 100 percent reliable at 100 kHz sample rate it seems.

Maybe tonight I will try running it at 100 kHz sample rate, on the transient screen, with the screensaver off, and see if it's still running in the morning.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: Rich on November 13, 2021, 09:47:14 PM
Hi MTCAT
If the display is updating a waveform, especially if it is scrolling it like a stripchart recorder, that will probably chew
up a bit of processor time.

It's possible the computer just isn't fast enough. Or maybe you just need faster video hardware.
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on November 15, 2021, 04:59:48 AM
Hi Rich,

I was able to run the program for 16 hrs at 100 kHz sample rate with a 240k sample dma_blk_size, with the screen saver off ("xset s off"), and the GUI on the mostly non-updating transient screen, however, at 200 kHz sample rate, it only ran for 90 minutes under the same conditions.

I then lowered the dma_blk_size to 200k samples, and the program ran for more than 12 hrs at 200 kHz sample rate !!, so that's a new record, it actually ran the battery dead (overnight). The 200k sample dma_blk_size is where I previously had the best success with 200 kHz sample rate (2.5 hrs, with screen saver on, and not leaving it on the transient screen all the time).

I guess I should run it at 100 kHz sample rate with 200 ksample dma_blk_size but maybe keeping the screen on all the time helps too.

Otherwise, could lower resolution to 800 x 600 ?, would have less pixels to draw and should also be a help I would think.

But maybe, if I leave the screensaver off, and leave it on the transient screen, might be able to squeak out 200 kHz sample rate reliably, 800 x 600 might give it a bit of a cushion too.

Will have to run some more tests I guess, am also going to run it on the smaller 10 inch LCD screen, which is what's going to be used in the field anyway.

Thanks,

David
Title: Re: TC3.8_4 Real Time Upgrade
Post by: MTCAT on December 05, 2021, 08:15:06 AM
Hi Rich,

Update, I think I got the little system going pretty good now at even 200 kHz sample rate, 200 k dma_blk_size, xset s off, and leave on transient screen, seems to run indefinitely at 200 kHz or 100 kHz sample rate !

Having problems with the DRACAL temperature sensor though (USB-TMP125), it needs a newer version of libusb-1.0 it seems, the DRACAL software won't compile. I did install digitemp 3.4 though, and it built just fine, so I have some simpler (and cheaper) temp sensors on the way that work with digitemp, so hopefully that will work, to have some real temperatures to report within the GUI.

On the GPS front, I can't buy the U-Blox EVK-M8T's at the moment (apparently there was a fire at a crystal factory in China U-Blox told me) so I'm building my own little GNSS timing solution with ZOE-M8Q from SParkfun, can configure it with a UART to USB adapter in Windows, and then use it for TinyCore with a TTL serial to RS232 serial level shifter. This is a lot cheaper than the EVK-M8T's as well.

Once I get the temp sensor and the new GPS setup going I'll have to modify the nice scripts you made for me to get the temperature readings out of a new csv formatted log file, and for the GPS, I would like to try to make use of new (to me anyway) NMEA sentences (GPGSV, GLGSV and GAGSV) to report the total number of GPS, Glonass and Galileo satellites, right now, we're using a NMEA sentence that reports the total number of GPS satellites only.

I also increased kernel.shmall (which Peter said should be done, but not sure if it's truly necessary) and set the "keepalive" time smaller but the latter unfortunately didn't solve the network problem (I don't think), when downloading libusb package I still had to get ping going in another terminal else it just stalled, hmm, problems to be fixed for another day, can always use USB to offload data as well.

So that's about it for me, THANKS SO MUCH, for all the help, I think this thread should be marked as SOLVED (and then some).

Thanks,

David
Title: Re: [Solved] TC3.8_4 Real Time Upgrade
Post by: Rich on December 05, 2021, 09:24:00 AM
Hi MTCAT
...  I think this thread should be marked as SOLVED ...
Done.  ;D
Title: Re: [Solved] TC3.8_4 Real Time Upgrade
Post by: xor on December 11, 2021, 08:44:33 PM
For single core processors this may be possible,
but for multi core processors RT core seems not possible!!!

Quote
https://github.com/rtic-rs/cortex-m-rtic/issues/204
It's not possible to share resources (static mut variables) between cores.
However, it is possible for one core to initialize another core's resources and
it's also possible to share static variables between cores.