Nov 22 2022
RPI3_MQTT broker_Node-Red dashboard
Posted by kll · In RPI · 211 Reads ·
for a short LOG (21.11.2022) / update (14.12.2022) / how to /
setup RPI3 for
* MQTT local broker
* Node-red dashboard
* PICO W Circuit Python programming
** target : PICO W / PMS1W MQTT webserver multitasking usb menu
use:
USB stick 8GB "Kingston Data Traveler 3.0"
with win 10 PC run:
SD Card Formatter 5.0.2 ( clean USB stick )
Raspberry Pi Imager v1.7.3
[ Raspberry Pi OS (32-Bit ) desktop ] [ Kingston ..] [ WRITE ]
on ethernet cable/TV/keyboard/mouse: RPI3 new OS first boot
setup linux user 'pi' password 'uptoyou'
name: RPI3
enable SSH VNC
set display--resolution VNC 1920*1080
reboot
ifconfig ( get IP ethernet) ( we not do a wifi setup as this RPI is a server ( with fixIP ))
from win10PC: ssh client login RPI ( RPI-IP, user password )
update
mkdir projects
optional:
sudo apt install -y obconf mc htop ethtool pastebinit neofetch
sudo nano /etc/dhcpcd.conf
interface eth0
static ip_address=192.168.1.103/24
static routers=192.168.1.1
static domain_name_servers=192.168.1.1 8.8.8.8 8.8.4.4
# as it will be a mosquitto and Node-Red web server not use WIFI ( but you can )
#interface wlan0
#static ip_address=192.168.1.203/24
#static routers=192.168.1.1
#static domain_name_servers=192.168.1.1 8.8.8.8 8.8.4.4
check space ( on 8GB stick ):
Filesystem Type Size Used Avail Use% Mounted on
/dev/root ext4 6.9G 3.4G 3.3G 51% /
# for use MQTT here as server:
sudo apt install mosquitto -y
sudo apt install mosquitto-clients
sudo nano /etc/mosquitto/mosquitto.conf
allow_anonymous false
password_file /etc/mosquitto/pwfile
listener 1883
sudo mosquitto_passwd -c /etc/mosquitto/pwfile u213S31
p213S31
p213S31
reboot
nano .bash_aliases
alias mqttsub='mosquitto_sub -h localhost -u "u213S31" -P "p213S31" -v -t "P213/#"'
# for use also python with MQTT:
sudo apt install python3-pip
pip3 install paho-mqtt
# add want use '.env' in python ( like in Circuit Python )
pip3 install python-dotenv
export PATH=”$PATH:/home/pi/.local/bin”
# try to get nodejs 16
curl -sL https://deb.nodesource.com/setup_16.x | sudo -E bash -
sudo apt install -y nodejs
#find v16.18.1 Npm 8.19.2
# install node red 3
bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)
# enable Node-Red autostart
sudo systemctl enable nodered.service
# enable Node-Red login ( here NO ) or projects ( here YES)
node-red admin init
reboot
# install sqlite
sudo apt install -y sqlite3
sudo apt install -y sqlitebrowser
cd .node-red
npm install node-red-node-sqlite ( run 9min on RPI3 )
reboot
# LAN browser open Node-Red
http://192.168.1.103:1880
NR: make project
PICO_W
NR: [manage pallette][install]
node-red-dashboard
NR:( check in node-menu if find:
network: mqtt in / out
storage: sqlite
dashboard: ...
#________________
RPI desktop
install from recommended software
Mu IDE
while on PC i use now v1.2.0
on RPI only get old 1.0.3 ( but it works )
check space ( on 8GB stick ):
Filesystem Type Size Used Avail Use% Mounted on
/dev/root ext4 6.9G 3.9G 2.8G 59% /
RPI system install finished and we can setup PROJECT:
unzip the project files into
/home/pi/projects/PICO_W/
# now get project files: PICO W with Circuit Python CP800b4 ( upgrade CP800 )
adafruit-circuitpython bundle .zip
adafruit-circuitpython * .uf2
flash_nuke.uf2
( foamyguy-circuitpython-nvm-helper-7.x-mpy-1.0.2.zip ) for the unused NVM test code
# my OLD python tools ( mqtt client send / mqtt client get _ save data )
# 3(4) terminal windows:
-1-
mqttsub
-2-
python mqtt_client.py ( it now uses .env file )
-3-
python mqtt_sub_power_meter.py ( it now uses .env file )
# ( optional terminal )
-4-
cat /run/shm/PowerMeter.csv
NEW tools update with LOCAL REMOTE BRIDGE
.env
pls adjust your online broker credentials ( and local broker settings )
python_tools\local_client_subscribe.py
python_tools\remote_client_subscribe.py
python_tools\remote_client_bridge.py
after all works
python_tools\start_bridge
check if path info is same as YOURS
need to
chmod +x start_bridge
make executable
python_tools\start.desktop
check if path info is same as YOURS
copy to your desktop for manual start
or to ~/.config/autostart/
for auto start bridge
# latest version of my PICO W code...
-1- use PICO_W board ( optional connect [reset] button pin 28 GND pin 30 RUN )
-1.1- powerup ( press [bootsel] while USB connect )
-1.2- copy flash_nuke.uf2
-1.3- copy adafruit-circuitpython_xxx.uf2
-1.4- see drive circuitpython, copy project
/lib/
*.py
( and ok overwrite code.py and settings.toml )
pls edit settings.toml first and copy to CIRCUITPY drive
adjust WiFi cred. / broker cred. / REMOTE broker cred.
-1.5- start Mu editor in Circuit Python mode
open file board code.py
open serial REPL
# Node-Red config at: http://192.168.1.103:1880/
from /projects/PICO_W/NR_backup/
take
RPI3_Flow1_flows.json
RPI3_Flow2_flows.json
RPI3_Flow3_flows.json
and import to
Node-Red Flow1 and Flow2 and Flow3 resp.. [deploy each]
update:
RPI3_Flow4_flows.json
for the S31 AC power trend and operation
update:
make add Node-Red project
PICO_W_REMOTE
import deploy
NR_backup\PICO_W_REMOTE\RPI3_Flow1_flows.json
edit mqtt broker settings for YOUR online broker and deploy
( when the python bridge is running should see record and trend in dashboard )
update: now the NR config is ONE FILE, the whole flows.json
import deploy reboot
the mqtt need a broker config once: [192.168.1.103]
and in security use
user: u213S31
password: p213S31
add manually as that is not copied by the Flow export/import
( deploy and now the node: 'mqtt in' should go green )
for the SQLite:
it is default to RAMDISK /run/shm/ ( here now 460 MB free )
minimal need to press the prepared node:'INJECT "create table mqtt_in ...' once
so if the mqtt stream starts it will be logged to database, but a reboot works too to create the database and table.
( now you could check if the file exists or use the sqlite-browser to open it )
new database (location) is:
/home/pi/projects/PICO_W/data.sqlite3
later should copy the OS system ( from uSD or USB stick )
to a USB SSD drive ( that can take the database heavy rewrites, while a uSD or USB stick NOT )
and store/ setup also the historic database to it..
( /run/shm/ is small and cleared with reboot )
well, there could be a combination like:
* a fast database in ramdisk and
* a cron job to add its data to SSD drive every hour?
a simple thing, but i forget and so better write it down here,
for the dashboard, i like the dark theme, custom colors..
where that is stored?
not see it in settings and it is not copied with the flow.json exports
but it is in flows.json as ui_base node, so better copy the whole thing.
on RPI4 could use also the USB3 M.2 SSD ( my fastest system )
( but i would not do that, it feels wasted for that small mqtt /sqlite job
that is why i use the old RPI2 and now RPI3 and not the RPI4 here )
but
usually i use this RPI3 with USB SSD
( and not with USB 8GB DataTraveler 111 )
anyhow boot from SSD / update /
and follow THIS manual again ( also for check )
WOW
this is great!
so why i work on that slow small usb stick for this project instead with the fast USB SSD system?
because i need to do following:
-1- change passwords / private data
-2- on win PC win32-Disk-Imager burn a copy of the USB-stick image
-3- zip it from 8GB to 2.8GB
-4- publish the system ( G-Drive ) ( so my friend not have to follow this setup, just burn a uSD card... )
slow chrome browser in RPI?
try settings/system/
use hardware acceleration when available [NO]
now we have on RPI3 installed SQLite3 and SQLite_browser app and SQLite for Node_Red ( internal )
Node-Red is creating ( auto ) and writing to a database ( if mqtt comes in to mosquitto )
Node-Red reads the database to show HIST LIST and HIST TREND
now here a feature from the installed SQLite3,
it has a command line interface:
$ cd ~/projects/PICO_W
$ sqlite3 -header -csv data.sqlite3 "SELECT * FROM mqtt_in;" > dbexport.csv
will export the database to a Excel file.
- - as RPI3 is a headless server i not installed office tools,
but with SSH SFTP copy the file to PC and show it in Libre Office Calc
