Articles Hierarchy

Articles Home » RPI » Python3_Signal_Filter_&_Line_Chart


i did it again, i will not allow this great tool to die out!

+ show a digital filter by Line Chart
+ + use command line parameters
+ + + add 2 sinus signals
+ + + + how to develop with python
+ + + + + python program use setup file
+ + + + + + output CSV and HTML with webserver
+ + + + + + + download and start
+ + + + + + + + just the web-job template

possibly you not remember that old chart recorders ( like this ?can still buy? from yokogawa )

but when i started arduino and analog measurement ( or simulation )
in a text terminal ( arduino IDE serial monitor )
i used something like this.
it is easy, free, can see it from some distance on the computer monitor ( or phone ) while you play hardware..
as now at the RPI 4um was a question for python3
+ show a digital filter by Line Chart
( signal processing ) i wanted to show a digital filter ( as play tool ).
with this
chmod +x
./ ( or python3 )

now on RASPBIAN Desktop and with PYTHON3 there would be many ways to make a graph
( a running "oscilloscope" like window i not that easy )
but this tool here also works via PUTTY SSH

+ + use command line parameters
now, while the filter is a ONE LINER
the signal simulation and the LINE CHART show makes it a project
and adding more and more options...
give it a default python3 layout
i now want ( for my training ) even add the command line parameter thing ( args / opts )
i just started with this when i did it for a bash tool raspi-info
now here again for python3.
( a little testprogram i got from here, my test code here )
the new MY_FILTER.PY with -s 10 and -h and -v looks like:

+ + + add 2 sinus signals
as this filter can reduce noise, it is a low pass filter,
it will reduce higher frequencies more as lower,
but also will delay the lower once.
to get a idea about the filter reaction i now want include add 2 sinus signals.
and the peak thing can be switched on by [ -p ]
the noise OFF only by [ -n 0.0 ]

because of the many switches / options i needed to
redo the print line ( chart + data ) and more help in the infolines above the line chart graph
new code

+ + + + how to develop with python
basic hint on:
working with PYTHON
- as beginner ( and also alzheimer candidate ) support on errors is essential, so i can only recommend to use IDLE3
- for projects where you need many open files ( like FLASK webpage..) that file handling is primary, the error thing is anyhow supported by flask / werkzeug ( file save / browser refresh )
should use ( now preinstalled ) GEANY
- for remote work using PUTTY / SSH ( i use bitvise )
and have 2 open windows and use
( this is how i do this little project / with only one file )
- 1 nano
- 2 ./
- or a combination of good PC local editor
( what can per SFTP work on remote files ) notepad ++
and again the putty window for execute / test.

- the THONNY python IDE i use rarely,
the internal shell AND the /tools / open system shell / not help,
but in combination with the desktop switching ( and again a lxterminal for test ) it works. The real project features not tested.

python lives from the INDENTATION
not use:
if ( we love RPI ) then { use it } or { toss it }

it is by structure:
if ( we love RPI ):
<-tab->use it
<-tab->toss it

( HA? next problem, difficult to show python in web pages, HTML kills exactly that unless you can use [code] [/code] )
so there must be 2 indentations by spaces or better tabs
and that makes it short in writing, good in reading, difficult in error checking.
from here

best is you have a editor showing the tabs or spaces.
one ( even its little headache ) is mcedit / the internal editor of mc

and, that also works same in PUTTY
( so even you not like its operation, just use it to check on spaces/tabs problems )

this BLOG here is primary a LOG of my work,
and i try to help beginners / with this web site / and with the work @4um /
this here is a mix of the
JOB: digital signal filter
learning python programming:
+ + python3 basic program structure
+ + HMI : CLI : parameter and help
+ + development tools : editors..
so what could be the next step for me as beginner?

+ + + + + python program use setup file
learn how to program setup by user / setup file.

first run? not find setup file
make path
write setup file: /home/pi/.config/kll/my_filter
[{'status': 'default', 'vertxt': 'rev 0.2.0'}, {'status': 'user', 'vertxt': 'rev 0.2.0 u'}]

[{'status': 'default', 'vertxt': 'rev 0.2.0'}, {'status': 'user', 'vertxt': 'rev 0.2.0 u'}]

with this test_setup code

here i use
import pickle


that is very easy, can do not only array, also classes..
but it is a record file not readable.
other ?better? way would have been JSON record..and readable file.
i try this now in,
means to rewrite all? or keep old variable and
write functions for copy to from old settings variables, anyhow takes a day..
when the program first run and not find the file it creates that ~/.config/kll/ dir and
makes a [{default}, {user}] record set and saves that to a ( unreadable) file " ~/.config/kll/my_filter"
even now you would edit that record in the program, it will not change anything unless you delete that file.
so, that local variables are disabled from the header with #
and later after READ or CREATE set by the record.
- - i failed doing it with set in a subroutine,
even using a global declaration, so its in the main again, just below the setup file handling. // i really need to learn CLASSES //
+ + the data thing works nicely also over the pickle file write read!
{'aname':'text', 'abool':True, 'areal':1.23}
now that would be just academic ( no real progress for the program )
or even bad as i write files in the user space...

as the user changes that variables at runtime,
by using the command line parameters
like: '-S' enable the 2 added sinus signals and possibly [-j 1.0 -J 0.1 -k 2.0 -K 0.5 ]
i want that it is possible to save that to the user record by adding a
[ -m] ( to memory )
with the result that when user next time use the program he can recall those settings by
[ -M ] (from memory)
( like the M keys on a Calculator )
as i keep 2 records, no factory reset required,
without any CLI parameter always use default
( local variable < recordset [0] < from file < from first run < from def make_my_default_opts():
here i run into the PYTHON shallow copy problem!
and play long time to understand, need:
import copy

other point was the program flow so it is possible to do
./ -M -s 15 -m
means take user setting, change a parameter, save user setting
so next call
./ -M
will be 15 lines long ( and use also other old user settings )
./ -m
will reset the user settings to default

i am happy about that concept, check out this code

+ + + + + + output CSV and HTML with webserver
now back to the program features,
urgently need some new function:

+ until now all we do is simulating,
but as this is about signals, there is ONE live signal in RPI what not requires to fiddle with hardware,
the CPU temperature
[-t --tempon] with this the noise signal is ignored.
i found 2 codes to read that system message by python at the forum, both not work,
its little bit tricky to make it a float.
from subprocess import PIPE, Popen # for read cpu temp

def get_temp():

signal = -5.0
process = Popen(['vcgencmd', 'measure_temp'], stdout=PIPE)
output, _error = process.communicate()
tempstring=outstr[outstr.index('=') + 1:outstr.rindex("'")]
signal = float(tempstring)
return signal

+ + save data to csv file ( and use later by Libre Office Calc )
[ -o test] creates a test.csv file ( at current location )
+ with headerline of the enabled columns
+ with value lines format .00
+ all comma separated

rev 0.2.4 code

+ + + save the whole thing (output) to a HTML content file
( remember multiple " " not work in HTML, try "_" to format the Line Chart for this )
and serve it with FLASK

here first some remarks to the web thing:
-1- nowadays it is often restricted by ISP ( Internet Service Provider )
you can enable a port forward in your router
( to your RPI_IP running a webserver ) still from outside (WWW) ( with your public IP (of your router )) you can not connect anymore.
( well there are the offers from many companies to tunnel your data
to their servers )
-2- this leaves the LAN / wifi network of your home.
you can ( just for learning ) install
LAMP ( Linux Apache MySQL, Php )
and a CMS ( Content Management System )
-2a- i doubt you run the RPI as a server ( like 24/7 on a UPS )
-2b- and 90% it is not even for experimenting learning ... users just think all that is necessary

now i did some great things:
my MAKER BLOG ( a BLOG CMS incl. a Process Control System )
just using FLASK
( so there is no installation required, no server running, until you start a 5 line python program
and the RPI can be reached from the LAN browsers.

learning LAMP and CMS might get you a job in old fashioned IT bussines,
but it is not the future.

but lets go back to the basics, if you need to get some sensor data in a browser page ( on same RPI ) you not even need a server,
from python create a file and call it
open the browser and find that file.

ha, that would be a stupid webpage?? NO,
HTML and scripting still works.. just no server requests possible.
long time ago i play this local way ( text editor and browser )
to develop a different way of menu ( i called it info menu )
( no, it is so old, windows tiles and dashboards are not available / known at that time )
see here
and it's fast like a video game.

and about learning, if HTML file or php or flask or ...
some HTML5 and scripting knowhow is required.
-a- so the very start could be any editor making that index.html file.
-b- create it by python ( like for overwrite )
and fill it with some sensor data
-c- serve it to the LAN by FLASK or NODE..

so with
[-w --www] can enable the index.html output
overwriting, autorefresh, minimal HTML header,
i started in desktop the browser ( and failed to find/enable the file menu )
but could type
or: desktop filemanager and double click on index.html also works
or: terminal, cd to my project, type:
chromium-browser index.html

now that is the idea,
-a- if you start
./ -w
for the first run you get:

pi@RPI3:~/projects/py3_digital_filter_first_order $ ./ -s 2 -w
my_filter (rev 0.2.5e) : CLI: [ -s 2] samples
operation: quit: [ctrl][z] / hold: [ctrl][s] / resume: [ctrl][q]
filter use: F = Fin * 0.10 + F * 0.90 | Fin from 50.00 +- Fn (noise) 10.00 | sampling 10.00 Hz
CLI: filter A [ -A 0.10] | noise [ -n 10.00] | sampletime [ -f 0.10]
Line_Chart: F (o) Fin (+)

|_________|_________|_________|_________|+_______o|_________|_________|_________|_________|_________|F 49.2_Fin 41.6_Fn -8.4
|_________|_________|_________|_________|________o|___+_____|_________|_________|_________|_________|F 49.7_Fin 54.3_Fn 4.3
you can open local browser at file:///home/pi/projects/py3_digital_filter_first_order/www/templates/index.html
OR to start flask server, in a new terminal type:
stop with [ctrl][c]
now from any LAN PC browser can call
pi@RPI3:~/projects/py3_digital_filter_first_order $

and from PC:

the server you should start from a extra terminal
cd www
( stop with [ctrl][c] )

and you can play more with the
./ until there is something you want show,
call it again with
./ -w
and the index.html is updated,
and the browsers will follow latest in 30 sec.

+ + + + + + + download and start
rev 0.2.5 this code
! note: this tool will make following changes to your system:
-1- at first start it will make the user setting file
-2- if you use the [-o mycsv ] switch it will make the file
-3- if you use the [-w] switch add get

when you click on above code a webpage with name
my_filter_py.txt opens. ( my server not allow .py , not even .py.txt files)
and it is little bit long, so <'text select'><'copy'><'paste'> is not too nice.
why not open a terminal:
mkdir ~/projects
mkdir ~/projects/my_filter
cd ~/projects/my_filter
# now here we try to use wget, can use from context menu <'copy link location'>
mv my_filter_flask_py.txt # can use smart bash, type my[tab]
chmod +x

+ + + + + + + + just the web-job template update 2018_04_26
i find the idea to mis-use the powerful FLASK for this ( serving a semistatic index.html file )
very good.
If you would like to go into Flask, Werkzeug, Jinja2, you will have fun!
But the beginner actually want the text or sensor data on the LAN
without learning python lists to html templates...
so i copy that filter program down to the webserver job,
a template to bring some small things online.
! there is no need to install anything ( i started on RASPBIAN Desktop )
! besides PYTHON 3 you not need to learn anything for the first start, but for further work
some HTML.. might be good.

this is a python3 template to easy serve your print text or data with FLASK
you can change:
html_title= KLL
html_header= template
html_loopdiv= <'div style="background-color:lightblue"'>

sampling time=10.00sec

on Friday 27 April 2018 06:05:05 VAL = 123.46
on Friday 27 April 2018 06:05:15 VAL = 123.46
on Friday 27 April 2018 06:05:25 VAL = 123.46

you can open local browser at file:///home/pi/.../www/templates/index.html
OR manually start flask server, in a new terminal type:
stop with [ctrl][c]
now from any LAN PC browser can call

* Running on (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger pin code: 324-033-258 - - [27/Apr/2018 06:11:55] "GET / HTTP/1.1" 200 - - - [27/Apr/2018 06:12:25] "GET / HTTP/1.1" 200 -
[1]+ Stopped ./www/

with this code