Articles Hierarchy

Articles Home » RPI » RPI3_MQTT broker_Node-Red dashboard

RPI3_MQTT broker_Node-Red dashboard

we come from here
for a short LOG (21.11.2022) / update (14.12.2022) / how to /

setup RPI3 for

* MQTT local broker
* Node-red dashboard
* PICO W Circuit Python programming


** target : PICO W / PMS1W MQTT webserver multitasking usb menu

use:
USB stick 8GB "Kingston Data Traveler 3.0"

with win 10 PC run:
SD Card Formatter 5.0.2 ( clean USB stick )

Raspberry Pi Imager v1.7.3
[ Raspberry Pi OS (32-Bit ) desktop ] [ Kingston ..] [ WRITE ]


on ethernet cable/TV/keyboard/mouse: RPI3 new OS first boot
setup user password
name: RPI3-32
enable SSH VNC
set displayresolution VNC 1920*1080

reboot
ifconfig ( get IP ethernet) ( we not do a wifi setup as this RPI is a server ( with fixIP ))


from win10PC: ssh client login RPI ( RPI-IP, user password )
update

mkdir projects
optional:
sudo apt install -y obconf mc htop ethtool pastebinit neofetch

sudo nano /etc/dhcpcd.conf
interface eth0
static ip_address=192.168.1.103/24
static routers=192.168.1.1
static domain_name_servers=192.168.1.1 8.8.8.8 8.8.4.4

#interface wlan0
#static ip_address=192.168.1.203/24
#static routers=192.168.1.1
#static domain_name_servers=192.168.1.1 8.8.8.8 8.8.4.4

check space ( on 8GB stick ):
Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/root ext4 6.9G 3.4G 3.3G 51% /



# for use MQTT here as server:

sudo apt install mosquitto -y
sudo apt install mosquitto-clients

sudo nano /etc/mosquitto/mosquitto.conf

allow_anonymous false
password_file /etc/mosquitto/pwfile
listener 1883


sudo mosquitto_passwd -c /etc/mosquitto/pwfile u213
p213
p213

( for new version with S31 Tasmota used u213S31 p213S31 )

reboot

nano .bash_aliases
alias mqttsub='mosquitto_sub -h localhost -u "u213" -P "p213" -v -t "P213/#"'
new version:
alias mqttsub='mosquitto_sub -h localhost -u "u213S31" -P "p213S31" -v -t "P213/#"'


# for use also python with MQTT:

sudo apt install python3-pip
pip3 install paho-mqtt

# add want use '.env' in python ( like in Circuit Python )
pip3 install python-dotenv
export PATH=”$PATH:/home/pi/.local/bin”



# try to get nodejs 16
curl -sL https://deb.nodesource.com/setup_16.x | sudo -E bash -

sudo apt install -y nodejs

#find v16.18.1 Npm 8.19.2

# install node red 3
bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)

# enable Node-Red autostart
sudo systemctl enable nodered.service

#( enable Node-Red login NO or projects YES)
node-red admin init

reboot
# install sqlite

sudo apt install -y sqlite3
sudo apt install -y sqlitebrowser

cd .node-red
npm install node-red-node-sqlite
( run 9min on RPI3 )

reboot

# LAN browser open Node-Red
http://192.168.1.103:1880

NR: make project
PICO_W

NR: [manage pallette][install]
node-red-dashboard

NR:( check in node-menu if find:
network: mqtt in / out
storage: sqlite
dashboard: ...

#________________
RPI desktop
install from recommended software
Mu IDE
while on PC i use now v1.2.0
on RPI only get old 1.0.3 ( but it works )

check space ( on 8GB stick ):
Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/root ext4 6.9G 3.9G 2.8G 59% /




# now get project files: PICO W with Circuit Python CP800b4
( upgrade CP800b5 )
adafruit-circuitpython bundle .zip
adafruit-circuitpython * .uf2
flash_nuke.uf2
( foamyguy-circuitpython-nvm-helper-7.x-mpy-1.0.2.zip ) for the NVM test code

# my python tools ( mqtt client send / mqtt client get _ save data )
# 3(4) terminal windows:
-1-
mqttsub
-2-
python mqtt_client.py ( it now uses .env file )
-3-
python mqtt_sub_power_meter.py ( it now uses .env file )
# ( optional terminal )
-4-
cat /run/shm/PowerMeter.csv


update with LOCAL REMOTE BRIDGE
.env
pls adjust your online broker credentials ( and local broker settings )
python_tools\local_client_subscribe.py
python_tools\remote_client_subscribe.py
python_tools\remote_client_bridge.py
after all works
python_tools\start_bridge
check if path info is same as YOURS
need to
chmod +x start_bridge
make executable
python_tools\start.desktop
check if path info is same as YOURS
copy to your desktop for manual start
or to ~/.config/autostart/
for auto start bridge



# latest version of my PICO W code...

-1- use PICO_W board ( optional connect [reset] button pin 28 GND pin 30 RUN )

-1.1- powerup ( press [bootsel] while USB connect )
-1.2- copy flash_nuke.uf2
-1.3- copy adafruit-circuitpython * .uf2
-1.4- see drive circuitpython, copy project
/lib/
.env
*.py

-1.5- start Mu editor Circuit Python mode
open file board code.py
open serial REPL


# Node-Red config at: http://192.168.1.103:1880/
from /projects/PICO_W/NR_backup/
take
RPI3_Flow1_flows.json
RPI3_Flow2_flows.json
RPI3_Flow3_flows.json
and import to
Node-Red Flow1 and Flow2 and Flow3 resp.. [deploy each]
update:
RPI3_Flow4_flows.json
for the S31 AC power trend and operation


update:
make add Node-Red project
PICO_W_REMOTE
import deploy
NR_backup\PICO_W_REMOTE\RPI3_Flow1_flows.json
edit mqtt broker settings for YOUR online broker and deploy
( when the python bridge is running should see record and trend in dashboard )



the mqtt need a broker config once: [192.168.1.103]
and in security use
user: u213 ( u213S31 )
password: p213 ( p213S31 )
add manually as that is not copied by the Flow export/import
( deploy and now the node: 'mqtt in' should go green )

for the SQLite:
it is default to RAMDISK /run/shm/ ( here now 460 MB free )
minimal need to press the prepared node:'INJECT "create table mqtt_in ...' once
so if the mqtt stream starts it will be logged to database.
( now you could check if the file exists or use the sqlite-browser to open it )
dir /run/shm/

later should copy the OS system ( from uSD or USB stick )
to a USB SSD drive ( that can take the database heavy rewrites, while a uSD or USB stick NOT )
and store/ setup also the historic database to it..
( /run/shm/ is small and cleared with reboot )
well, there could be a combination like:
* a fast database in ramdisk and
* a cron job to add its data to SSD drive every hour?

a simple thing, but i forget and so better write it down here,
for the dashboard, i like the dark theme, custom colors..

where that is stored? no idea,
not see it in settings and it is not copied with the flow.json exports

on RPI4 could use also the USB3 M.2 SSD ( my fastest system )
( but i would not do that, it feels wasted for that small mqtt /sqlite job
that is why i use the old RPI2 and now RPI3 and not the RPI4 here )



but
usually i use this RPI3 with USB SSD


( and not with USB 8GB DataTraveler 111 )
anyhow boot from SSD / update /
and follow THIS manual again ( also for check )

WOW



this is great!


so why i work on that slow small usb stick for this project instead with the fast USB SSD system?
because i need to do following:
-1- change passwords / private data
-2- on win PC win32-Disk-Imager burn a copy of the USB-stick image
-3- zip it from 8GB to 2.8GB
-4- publish the system ( G-Drive ) ( so my friend not have to follow this setup, just burn a uSD card... )



slow chrome browser in RPI?
try settings/system/
use hardware acceleration when available [NO]


now we have on RPI3 installed SQLite3 and SQLite_browser app and SQLite for Node_Red ( internal )

Node-Red is creating ( auto ) and writing to a database ( if mqtt comes in to mosquitto )
Node-Red reads the database to show HIST LIST and HIST TREND

now here a feature from the installed SQLite3,
it has a command line interface:

$ cd ~/projects/PICO_W
$ sqlite3 -header -csv data.sqlite3 "SELECT * FROM mqtt_in;" > dbexport.csv

will export the database to a Excel file.
- - as RPI3 is a headless server i not installed office tools,
but with SSH SFTP copy the file to PC and show it in Libre Office Calc