How to upgrade the version of Python running on EPIC or RIO

SSH
1

We have had a few customers want to do ‘more’ with Python on EPIC and RIO.
The problem they quickly come across is that the version on each is a little dated for their needs.
They specifically need to install either Pandas or a newer version of Numpy.

The solution is to install a newer version of Python alongside the existing version. Note that you can’t just update the version already on the EPIC - it must remain the same as it is used by many other packages on the EPIC/RIO and updating it will break those packages. (Hint, you will need a paperclip to recover).

The key is the ‘altinstall’.

Please Note: As new versions of Linux packages are release, the dependencies of those packages change, this can cause build processes to break. (Which is out side of Opto’s control).

On top of that when we release new groov firmware versions, we make changes to our package repo. This is in our control and we generally only add packages that we know work Ok with our Yocto build of Linux.

Bottom line of those two variables is sometimes the packages don’t align any more and so something that worked months ago, no longer works.

Ok, so that’s the why, how do we install pandas and numpy with a newer version of Python?

Here are the first steps to install Pandas which has Numpy as a prerequisite.
Install the Python prerequisites…

sudo apt-get update
sudo apt-get install libz-dev
sudo apt-get install libffi-dev
sudo apt-get install openssl-dev
sudo apt-get install libreadline-dev
sudo apt-get install libpcap-dev

Get a newer version of python.
NOTE! This may not be the newest version, but it is newer than the one installed from the factory, and it is one we have tested and found to work. You are welcome to try/test newer versions, but if it fails, the best thing to do is a firmware update which will erase all changes made in shell and start over again from fresh.

wget -O ~/python3.9.15.tgz https://www.python.org/ftp/python/3.9.15/Python-3.9.15.tgz

Untar it.

tar -zxvf python3.9.15.tgz

cd into the directory so we can configure and make it. Note the make process will take around 20 minutes.
Note the use of the altinstall command!

cd Python-3.9.15/
./configure
sudo make -j4 altinstall

(Drop the -j4 when on a RIO)

NOTE! If you just want a newer version of Python, you can stop here. You are done.

This next step is very important! You must call out specific versions of numpy and pandas to build the packages.

sudo pip3.9 install numpy==1.23.0 pandas==1.4.4

After the build (which takes a few hours due to all the packages that are built) you can check your built and installed versions:

pip3.9 list

The result of this command on my groov is as follows:

Package         Version
--------------- -------
numpy           1.21.2
pandas          1.3.5
pip             21.2.3
python-dateutil 2.8.2
pytz            2021.3
setuptools      57.4.0
six             1.16.

@torchard and I will try and keep these instructions updated as time goes by.

2 Likes
2

Hi, I’ve followed this to get python3.9 and pip3.9 working but they are only available with this procedure by running them with sudo. That has worked for testing but I’d like to run at startup under my shell user and my shell user can’t seem to access python3.9 then. I tried using crontab with sudo to set it up but it doesn’t seem to run, is there a way to get python3.9 then accessible under my normal user, so I don’t have to root account everything?
Here is the directory that resulted from your steps above, all owned by root and my localssh account even in that folder can’t run the files, here the output I get…

amissh@192.168.1.182:/usr/local/bin$ ls -l
total 12908
-rwxr-xr-x 1 root root      101 May  7 22:09 2to3-3.9
-rwxr-xr-x 1 root root       99 May  7 22:09 idle3.9
-rwxr-xr-x 1 root root      229 May  7 22:12 pip3.9
-rwxr-xr-x 1 root root       84 May  7 22:09 pydoc3.9
-rwxr-xr-x 1 root root 13195916 May  7 22:04 python3.9
-rwxr-xr-x 1 root root     3093 May  7 22:09 python3.9-config
amissh@192.168.1.182:/usr/local/bin$ sudo python3.9 -V
Password: 
Python 3.9.15
amissh@192.168.1.182:/usr/local/bin$ python3.9 -V
bash: python3.9: command not found
3

Mike, the problem you’re having running python3.9 as an unprivileged user might be caused by an incomplete PATH definition for the amissh user, but we need more information to know for certain.

When you invoke a program from the shell without providing a base path (relative, absolute, or ./), then the shell iteratively searches for the given executable in each directory in the PATH variable in order until there’s a hit.

In your post above, you wrote that this command succeeded with root and not with an unprivileged user,

amissh@192.168.1.182:/usr/local/bin$ sudo python3.9 -V
Password: 
Python 3.9.15
amissh@192.168.1.182:/usr/local/bin$ python3.9 -V
bash: python3.9: command not found

This would be expected behavior in the case that /usr/local/bin is in root’s PATH but not amissh’s PATH.

Can you please post the output of the following two commands run as amissh and root via sudo?

$ echo "$PATH"
$ sudo echo "$PATH"
4

I should update that top post with how to use it, but as @Indigo said, its about paths.
I would NOT recommend putting the new version in your system path - that will break much of groov.
Rather you need to just path out to the new version of Python when you run it either from a script or from cron. It does not need sudo.
We have dozens of customers running apps with the new version that they build, so its a non-issue to run as your shell user with the path explicitly called out each and every time. (The draw back of the altinstall).

5

I would NOT recommend putting the new version in your system path - that will break much of groov.

Agreed.

You can save yourself extra typing by adding an alias to /home/amissh/.bashrc

$ echo "alias p='/usr/local/bin/python3.9'" >> /home/amissh/.bashrc
$ source /home/amissh/.bashrc
$ p -V
Python 3.9.15

Understand that cron, the system, and other users won’t have this alias. An attempt to use this alias from cron, the system, or another user will result in command not found, so use the alias only for testing.

1 Like
6

Thanks for the help on this guys, good to know others are running python apps, and I’m sure I will be soon too. The python side is easy for me, but not so much the linux side, however, learning fast.

Here is my echo path outputs:


amissh@192.168.1.182:~$ echo "$PATH"
/usr/bin:/bin:/usr/sbin:/sbin
amissh@192.168.1.182:~$ sudo echo "$PATH"
Password: 
/usr/bin:/bin:/usr/sbin:/sbin

I’m fine with needing full paths to run python3.9 (and understand the importance to keep it explicit to not interfere with the built in versions). With that simple knowledge and a bit of playing now, I was able to get my local account to run my daq/logging app and it runs in crontab now as well as long as I reference that full path to launch python.
So, all running how I needed now, thanks for the tips and guidance!

2 Likes
7

Good day
I have a groov rio and I need the steps how to use python to program the inputs and outputs in the device

8

Have you checked out our developer site?

https://developer.opto22.com/pythonmmp/pythonpackage/#python-package-overview

9

Thank you I will follow up the information you share with me