Running Python Scripts at Startup

[xuraax]

I would like to run a Python script, which runs well from command line, directly after booting on a headless Pi.

I have read previous help on this topic but nothing seems to work for me including running the script with &, inserting a sleep 60 delay before the " python3 /home/tc/trial.py & " in the bootlocal.sh file. Even a simple python file consisting of an infinite loop which prints just a string to screen. Inserting " echo 'Hello There' " directly into bootlocal.sh also produces nothing. Clearly I am missing something quite basic.

Any help appreciated.
xuraax

[Rich]

Hi xuraax
Try adding the command to the end of  .profile  in your home directory.

[xuraax]

Hi Rich.
Thanks for your suggestion.
I can confirm that the script does run when invoked from the .profile file. However for some reason it does not operate in the same way as it does when run from the command line. I still need to analyse this properly.

I first thing I noticed was the following:

My script turns on an LED to signify that it is functioning and then goes into a loop to establish communication via the UART. Each attempt is confirmed by a print statement which signals the attempt number.

The strange thing is that even if a wait for several minutes before I SSH into the system when I am certain that the link has been established, the message that appears on the screen is as if the attempt in communication has just started.

My other question is: why did this script not work at all in bootlocal.sh?

Regards

[bmarkus]

It would be good to see your Python script and how is it called. Without information it is just guessing.

[xuraax]

It would be good to see your Python script and how is it called. Without information it is just guessing.

I have no problem with sending the script but I must warn you it is about 200 lines long and, coupled with the fact that I am not an expert programmer, it can get confusing. Just tell me how to proceed and i will comply.

The script is built around an old RPI to which I added a small 2 transistor circuit to the UART to interface over the K-line to my bike's ECU. Once logged on it collects all data and sends it to a csv file which I further process off line using Excel. The idea is to collect live data whilst on the move.

Regards

[xuraax]

After writing the above mail I realised I could demonstrate what is happening by writing a very simple script which I called trial.py and invoke that in the .profile file. The script simply consists of the following:

Code: [Select]

import time
i=0
while True:
    print("Sample no.: ",i)
    i+=1
    time.sleep (1)
   
In the .profile file I inserted

Code: [Select]

sudo python3 trial.py
Note that I need to use sudo in my own program since this is required to access the UART.

After saving and rebooting and SSHing into RPI one notes that the first thing that comes on the screen is "Sample no.: 0"

Keeping the script running and exiting SSH and then reentering again with SSH, the screen still comes up with "Sample no: 0" even if I allow some time to pass before reconnecting.
Also with the script is running and you do ^C the script appears to terminate with the correct error message. However in fact the script is still running in the background and doing

ps aux

shows that python3 trial.py is still running with its own job number.

[patrikg]

If i don't misunderstand your request, you could do it like this.

.profile is running when entering the shell.

Too keep the script running after exiting the shell you could add nohup to the script so the shell don't exit its child processes.   

https://linuxhint.com/how_to_use_nohup_linux/

[xuraax]

Too keep the script running after exiting the shell you could add nohup to the script so the shell don't exit its child processes.   

nohup is an interesting command but it does not achieve what I want. What I am looking for is the following:

1. when I power up my RPI, my script initiates and goes about its work collecting data loop after loop.
2. at some later point when I want to "look in" I would like to SSH into the RPI to see what is happening during that particular loop. if say at teh point of entry the variable in my trial.py happens to be i = 1000, I would like to see teh following on screen:

Sample no.: 1000

3. while here I could decide to interrupt my loop and stop the script completely or continue monitoring on screen the data being collected.

In the past, I have done something similar (but not with autostart) by using TMUX which allows you to look into a process by attaching or deattaching from that process.

[bmarkus]

Use 'screen'

[xuraax]

Screen appears to be another version of tmux.

what I tried was to insert in the .profile file:

Code: [Select]

screen -S mysession
sudo python3 trial.py


in the hope that, when I later ssh into the box, I can do:

screen -r mysession

to reconnect but this did not work because I had not deattached from "mysession" using

^A^D

after initiating the script.

Doing:

screen -ls

listed 2 still attached processes named "mysession" (why 2?)

I guess this method could work if I am able to somehow invoke ^A^D from the .profile file to deattach "mysession".

any ideas?

[FlyingDutchman]

Hi xuraax,

Just my 2 cents here, as I know almost nothing about Python.
I think you can execute Python code from bootlocal.sh (I don't see any reason why not), but I don't think you can write any output to a terminal or console. When bootlocal.sh runs, there is no tty connected to that process, afaik.

The easiest way I can see is to log your output to a file and then reading that file when you ssh into the system. If you want that fully automated, you could add

Code: [Select]

tail -f <FILE> to your .profile script.
To avoid the file growing until eternity, maybe you should also write some kind of cleanup mechanism that clears the whole file, or certain entries in the file.

Cheers.

[Rich]

Hi xuraax

... The script is built around an old RPI to which I added a small 2 transistor circuit to the UART to interface over the K-line to my bike's ECU. Once logged on it collects all data and sends it to a csv file which I further process off line using Excel. The idea is to collect live data whilst on the move.

So it sounds like you probably have the data collection working and now want to be able to display snapshots periodically.

My recommendation is you run your program in the background and do not print anything to the screen.
Print the data to a file (Data.txt) and overwrite the file whenever fresh data comes in.

Use a second file (Status.txt) initialized to 0 to synchronize accessing the file:
When you want to read the contents of Data.txt, set the contents of Status.txt to 1.
When your python program sees the read request and is done updating Data.txt, it sets the contents of Status.txt to 2.
As long as the contents of Status.txt remains equal to 2, your python program skips updating Data.txt.
When you see Status.txt has been set to 2, it is safe to read the file.
After you have read Data.txt, you set Status.txt back to 0 which tells your python program it is safe to resume updating Data.txt.

[Rich]

Hi xuraax
The following can be used to monitor the contents of Status.txt and display the contents of
Data.txt when your python program signals that it is safe to do so.

Code: [Select]

#!/bin/sh

DataFile="Data.txt"
StatusFile="Status.txt"

# Create Status file if it doesn't exist.
[ -f "$StatusFile" ] || echo 0 > "$StatusFile"

while true
do
sleep 2
status="`cat $StatusFile`"

case "$status" in
0) # Signal that we want to read DataFile.
echo 1 > "$StatusFile"
;;

1) # Wait for status to change to 2.
continue
;;

2) # Clear the screen, display the data, and reset StatusFile to 0.
clear
cat "$DataFile"
echo 0 > "$StatusFile"
;;
esac
done

[xuraax]

I think you can execute Python code from bootlocal.sh (I don't see any reason why not), but I don't think you can write any output to a terminal or console. When bootlocal.sh runs, there is no tty connected to that process, afaik.

Thank you for your suggestions. In fact my full script includes also an LED output to signal what is happening and one input to test the position of a push button. As you proposed I have removed all tty output and rely on the LED to tell me if the script has started, if connection has been established and then that data is being collected and stored to a file. The push button interrupts the loop and closes the data collection file and then the script itself.

This still did not work for me in bootlocal.sh but does work using the .profile file. Maybe the reason why the former method is not working for me could be because my script needs to use UART 0 which is probably being used by the system as console output during the booting process.

Anyway...I still would like to explore further the use of

screen

invoked from the .profile file and it seems that if I can manage to invoke

^A ^D

from here I may be able to achieve my aim. Indeed looking further into the commands available in

screen

I noted a command called

detach

which I had hoped would do the trick for me but alas no joy!

[xuraax]

The following can be used to monitor the contents of Status.txt and display the contents of
Data.txt when your python program signals that it is safe to do so.

Thank you Rich for your suggestions. I will consider it. Another possibility would be to save to a different .csv file every, say, 1000 samples from the script running in background and just open up a particular .csv when I log in with ssh.

I still would like to see if can "peek in" into my running script using "screen" if it is at all possible.