what is MEAN stack? Posted by kll on April 10 2016 14:17:47while i play with the processing js ( all just in one index.. file and no need for server... ) i was told for connect a arduino - usb - RPI - webserver.. "today you must have" MEAN stack instead of old fashioned LAMP server, with
LAMP := Linux, Apache, MySQL, PHP. and
MEAN := Mongo, Express, Angular, Node.
possibly its home is here? or here; differences? also see github and mongo db
what hit me was that mongo db is NOSQL.
anyhow i started first with unpacking my RPI2 and install it in my new weekend house
using shared WIFI from my neighbor. as it was not the latest release i needed some update/upgarde.
while checking setup i see that new program node-RED ( from terminal node-red )
and just tested what info was there: test with browser ( from my desktop PC: http://192.168.1.35:1880)
i learned that not only node.js is already installed in the latest RPI jessie, much more of ?that new world? is ready to play.
from ?IBM? see this
if you check here you find many ready functions you can import,
a- you should be in your node RED directory cd ~/.node-red
b- find the install command for the flow or node you like npm install node-red-node-mongodb ...Link or Link or Link or Link ( that one failed )
check in ls node_modules whats there.
c- start the node-red ( from terminal ) or node-RED from desktop
d- start the node-red site in any browser at your network: http://192.168.1.35:1880
and find the new installed nodes to play with.
but the help text of node-RED say also better do: sudo apt-get install npm
sudo npm i -g npm@2.x
what's npm? npm install express
npm install sockjs node -v !! v0.10.29 from 9.6.2014
also i see a interesting link about home control using exactly arduino and RPI with mango db node js...
i should try to compile a newer version of node.js, it might fail, or break node-RED..
opt v0.12.13 from 31.3.2016 cd Downloads
wget http://nodejs.org/dist/v0.12.13/node-v0.12.13.tar.gz ahm https://nodejs.org/dist/latest-v0.12.x/node-v0.12.13.tar.gz from 31.3.2016 might have been better or /dist/latest/node-v5.10.1-linux-armv7l.tar.xz i will try next see here
tar xvf node-v0.12.13.tar.gz
cd node-v0.12.13
./configure
make 1h40min
sudo make install
ok, node -v show v0.12.13, node-red still running, and its webpage at http://192.168.1.x:1880/ also ok.
its time for the first steps: following John Saunders
making a work directory with a package.json and server.js file
and a subdir /public/ with index.html and RPI logo image file
and start node ( from the work dir ) with sudo node server
give me
but first i get errors about the modules listed in the package file, after installing them it worked,
but now i see that i have several /node_modules subdirs, possibly my setup mistake.
while i use npm install the package.json file is checked and i see lots of warnings,
working on it step by step i put in lots more ( not really needed) info lines in it.
package.json: {
"name": "KLL-node",
"version": "0.0.2",
"description": "try node js server on RPI2",
"url": "kll.engineering-news.org/kllfusion01/articles.php?article_id=112",
"license": "CC-BY-3.0",
"repository": "kll.engineering-news.org",
"main": "server.js",
"dependencies": {
"express": "~4.13.4",
"morgan": "~1.0.1",
"body-parser": "~1.0.2",
"method-override": "~1.0.2"
}
}
for mongo db the info is very confusing, 5 hour compiling / what mostly fails / download pre-compiled from 2012 ...
can that work on jessie? / i read that for jessie it should be available for install, but not find where and how...
so i try one of that old ready things: widriksson
yes, not forget to unzip the downloaded file ( and be at that path / and be admin ( sudo su ) )
only then the commands from that manual will work.
after reboot i can still call it in RPI browser ( localhost:28017 ), so the install looks good:
when i try to play with it only get a text info about REST
from here i see: sudo nano /etc/init.d/mongodb
find that line and add --rest: DAEMON_OPTS="$DAEMON_OPTS --config $CONF" --rest
DAEMON_OPTS="$DAEMON_OPTS --config $CONF --rest"
and restart sudo /etc/init.d/mongod restart now ok
use node with mongo need install: npm install mongodb
when i look how to create a database with mongo i see that just use it, if not exist it creates automatically use mydatabase
from the CLI and open the CLI with mongo now that did not work for me, also not with sudo, but mongod does something ( bad?) ?start a new mongo instance and stop it again?
sorry, forget all above, i try again with a complete new setup:
2016-03-8-raspberry-jessie.img on 8GB SD
NODE
the mean part: check on node: node -v // 0.10.29
this time i want get the newest: ( see here ) cd Downloads wget https://nodejs.org/dist/latest/node-v5.10.1-linux-armv7l.tar.xz tar -xvf node-v5.10.1-linux-armv7l.tar.xz cd node-v5.10.1-linux-armv7l sudo cp -R * /usr/local/ sudo reboot
pi@RPI2:~ $ node -v
v5.10.1
pi@RPI2:~ $ npm -v
3.8.3
ok, now i copy my webpage, ( yes i made a backup b4 use Win32DiskImager ) to
/projects/node/public/
/projects/node/ ( but without the /node_modules/ directory ) npm install
get all in /node_modules/
pi@RPI2:~/projects/node $ sudo node server
Simple static server listening on port 4567 ^Cpi@RPI2:~/projects/node $ can stop with [CTRL][C]
ANGULAR
what i not mention until now is the meAn : from the Angular JS
i understand that there is nothing to install at the RPI,
you just make a web page and link to the Google Angular code project.
for the start i select a tutorial
and play with that code like here, i just download all, and save as index.html and start node server
EXPRESS
the mEan thing express we used already:
var app = express();
to get a better idea i fly over tutorial or tutorial
interesting special that generator use here
short form: sudo npm install express-generator -g
express myapp1
cd myapp1
npm install
DEBUG=myapp1:* npm start
gives me:
MONGO
like Mean, here we find mongo
this time i follow Link
( but also say it use mongo 2.1.1 from 2012 )
cd Downloads
wget https://github.com/tjanson/mongodb-armhf-deb/releases/download/v2.1.1-1/mongodb_2.1.1_armhf.deb
sudo dpkg -i mongodb_2.1.1_armhf.deb error libboost-dev
sudo apt-get install libboost-dev error
try libboost1.55-all-dev-1.55.0+dfsg-3
libboost1.55-dev--1.55.0+dfsg-3 from desktop add/remofe software tool
and try again:
pi@RPI2:~/Downloads $ sudo dpkg -i mongodb_2.1.1_armhf.deb
(Reading database ... 140203 files and directories currently installed.)
Preparing to unpack mongodb_2.1.1_armhf.deb ...
Unpacking mongodb (2.1.1) over (2.1.1) ...
Replaced by files in installed package mongodb-server (1:2.4.10-5) ...
Replaced by files in installed package mongodb-dev (1:2.4.10-5) ...
Replaced by files in installed package mongodb-clients (1:2.4.10-5) ...
Setting up mongodb (2.1.1) ...
Processing triggers for systemd (215-17+deb8u4) ...
Processing triggers for man-db (2.7.0.2-5) ...
pi@RPI2:~/Downloads $
sudo /etc/init.d/mongodb start sudo update-rc.d mongodb defaults
browser: localhost:28017 // ok, and now it not works from network!! good
show db version v2.4.10
again the REST thing
sudo nano /etc/init.d/mongod now wrong
sudo nano /etc/init.d/mongodb
DAEMON_OPTS="$DAEMON_OPTS --config $CONF --rest" in 3 lines! STILL NOT WORK change back
new way to do that: sudo nano /etc/mongodb.conf
add line
rest = true sudo reboot
you see, localhost:28017 and its links works,
and this time also the mongo
CLI also ok.
_________________________________________________________________
it is good but still old, i got a other link and i have to try mongodb-3-0-9-binaries-for-raspberry-pi-2-jessie
!warning: whats missing info at that page:
-a- download wget http://andyfelong.com/downloads/core_mongodb.tar.gz 409MB and > 1 hour wget http://andyfelong.com/downloads/tools_mongodb.tar.gz 12MB
-b- unzip an other 1100MB so i was short with my 8GB SD card.
-c- the unzip gives you all the mongo* executables IN SAME PATH!
so better start with a mkdir temp, cd temp
-d- i have a mongo running so the copy command not work! sudo service mongodb stop // KLL sudo mv mongo* /usr/bin // KLL the copy can be a space problem again
-e- as mongo was installed only a few of the changes in conf and system where required
ok, independent what tools you use, the functional structure for this job must be same:
-1- you need a database to store the data to
-2- you need a service handling the USB interface
- - reading LINES and interpret them and save them to database
- - send out commands ( to the arduino ( menu sketch ))
- - allow some diagnostic function * service running * data stream active *
- - auto start at boot
-3- you need a interface to read the collected data
- - a web server
- - a web page list style
- - a web page interactive control style
- - a export function
i think we got all the tools: lets start
JSON dataformat
a friend send me a arduino example for testing a json string with sensor data, for me to see what it looks like.
i try with arduino UNO, USB RPI, normal arduino install on RPI, later i change to a more general use as a loop over the analog ins, using also a basic arduino time what later will come from RTC.
i understand that that format should have big advantages on the programming side like for Mongo database storage.
well, for some slow application you can use any of that "FULL TEXT" data stream formats,
i still prefer CSV lines with numbers only, and a way to communicate ( on demand) about understanding what number means what, like tag name, tag desc, low high range, engineering unit, alarm / alarm limits... so you send numbers in 100.00 % thinking, and the ranging you do at the visualization side: like i do at my uPCS project and as it is industrial standard for DCS / PCS / PLC.
read by NODE
from here i see the first steps of node and serial interface.
and just get the arduino lines " to console "
also see node-serialport
using arduino code and node code
but that was nothing, just reading, now the tricky part about the understanding of the JSON string:
you have to convert the string to a JSON object by JSON.parse(serialstring)
as i used the string in the "named list" style ( i have no better word for it )
{"A0":"123","A1":"456" ....} i can use the object.A0 to call the values,
and i checked, a number "123" is still a number and can be multiplied... console.log(serialData.A0 + " mul 2.5 " +serialData.A0 * 2.5); funny that the * is math and the + is string concatenation,
but actually that's a lot of typing / coding, even a readable one.
but i want try to use the array style, and if possible
even get the info from arduino how many data it sends, to work on both sides just with a for loop.
in arduino i make a JSON list with CHANNELS, AIN , DESC, values and desc as array
in RPI serial JSON object can use the CHANNELS for the end of the for loop
over the array of desc and values.
and it looks like that
for this test / functional example i not included the TIME, just to keep it short, we can use it same way we did already,
using arduino code and node code
now we need to connect to mongo, make a database, make a collection, and save the arduino USB records.
as example we use the arduino lines type "named list" of all analog-ins from the first arduino code.
( but that does not matter here, as we just save the full JSON object / record and
mongo creates automatically the index "_ID".
this are the additional lines for the db connection and write operation.
but how to know it worked?
use the mongo CLI and
- check for database:
>show dbs
and select the one we just created
>use arduino
- check for collections
>show collections
- check for the records in our new "AINS" collection / table
>db.AINS.find()
if the list is longer must use
>it
you will find a lot of usefull commands there with help,
but nothing about how to delete a collection or even database??, well here is the big hammer: mongo arduino --eval "db.dropDatabase()"
and i checked with ls -la /var/lib/mongo* that the 2 arduino files are gone. ?no backup?
now here i only use the minimum code for get and store serial JSON strings:
you start from command line: node arduino_serial_mongo_min.js /dev/ttyACM0
port open:/dev/ttyACM0 Data rate: 9600 ^C and stop with [CTRL][C] and here the code.
when i start in a new directory i also make a package.json file and run npm install
+ + no warning...
+ + get all the /node_modules/...
- - but still a problem about a node-pre-gyp, even i already did a sudo npm install node-gyp -g aha, its a serial port npm thing
autostart a service
you need to start programs at boot usually with absolute path... like
/usr/sbin/node /home/pi/projects/MEAN/arduinostream/arduino_serial_mongo.js /dev/ttyACM0
well, that not works here. you need to prepare a startup file like
means you must go to the path where you developed the program ( with cd ) and start it as you would do from command line.
now you can do a service file at sudo nano /etc/init.d/service
and that calls the node program via the bash script
/home/pi/projects/MEAN/arduinostream/arduino_serial_mongo.sh
and don't forget sudo chmod 755 /etc/init.d/service
sudo update-rc.d service defaults
sudo /etc/init.d/service start
and check if it still works after reboot
BUT if you need to touch the arduino / by IDE... /
you need to stop the service first. i did but still could not access the arduino, so the service stop was not enough to stop node??
i removed the service completely and reboot, IDE upload OK.
need to find a better way to stop of the nodejs serial read line loop, i expect a process.on('exit' .. with a serial.close() could help,
but where and how to do??
i have for "make it a service with autostart" add my GUI tool ( using zenity )
but there are sure better tools outside... i will take a look at npm install pm2@latest -g from here
( sorry i need sudo for this )
there is a warning about incompatible dependency fsevents ( ok, that for OSX only )
but also seems to have a problem with jessie, need sudo pm2 startup systemd -u pi
but their GUI tool is not included / ? not for free /
first i make a start pm2 file
pm2 start process.yml pm2 show 0
and follow the info for the output logfile to see its active, or more easy pm2 logs 0
ok, now test the reboot pm2 show 0 ok mongo arduino
> db.AINS.find() ok
desktop arduino IDE terminal say nothing: NO ERROR NO DATA
but now, how to stop that service? pm2 stop 0 will stop the process ID 0 ( expect should start at reboot again )
arduino IDE terminal OK
arduino IDE upload OK pm2 start 0
now i need to go over the js code again, clean up the many diagnostic outputs... pm2 stop 0 pm2 delete 0 will remove the process ID 0 ( expect should NOT start at reboot ) get the newest ardunio IDE on RPI
for a auto start using systemd i found at raspberrypi.org/forums
i need to play with that later.
WEB SERVER
show Mongo db data style i find here a introduction, ups that's a big step one, i just fly over it, this one sounds about right and also use this
and here already with the listview of the arduino data:
and the code segments to do it:
app.js
var mongo = require('mongodb');
var monk = require('monk');
var db = monk('localhost:27017/arduino');
....
// Make our db accessible to our router
app.use(function(req,res,next){
req.db = db;
next();
});
/views/arduinodata.jade
p #: yyyy-mm-dd hh:mm:ss|A0 |A1 |A2 |A3 |A4 |A5
ul
each rec, index in arduinodatalist
li= index + ': ' + rec.time + '|' + rec.A0 + '|' + rec.A1 + '|' + rec.A2 + '|' + rec.A3 + '|' + rec.A4 + '|' + rec.A5 + '|'
/routes/index.js
/* GET DATA list page. */
router.get('/arduinodata', function(req, res) {
var db = req.db;
var collection = db.get('AINS');
collection.find({},{},function(e,docs){
// console.log(docs); // works
res.render('arduinodata', {
"arduinodatalist" : docs
});
});
});
i need a code editor ?
ok, back to the mongo database
i play a few days on the arduino side for a motor application
- one Dout start stop a 24vdc motor via a transistor
- a Adafruit_INA219 reads the supply voltage and the motor current
( i could not play with the hardware, just the code)
i did a software Motor Control Center
+ USB menu start stop
+ scheduler start motor 4 times per day for X seconds
+ at stop a (JSON) report shows start time, stop time, min max current while START and RUN time
+ a START time masks inrush current for different fuse action at START and RUN status
+ a recursive digital filter first order filters the current for a FUSE logic ( make it "slow blow" )
+ a alarm function should WARN prior to any FUSE action
now here i run into a double problem,
-a- the arduino get full from all that text messages, even using F("") stored at ROM
-b- the USB gets all that traffic of long text
-c- the node.js will filter the TEXT lines ( as GARBAGE ) to LOG
-d- the JSON also long text lines will be stored to database
as i want learn more about that NOSQL way and JSON i think of a new way to do that, efficient on both sides:
idea comes from the thinking in a distributed process control system
where the controller ( something like a PLC, but bigger slower / more for analog / PID control as for fast digital PLC action )
never gets that names and texts... it communicates with the logical POINTS ( a analog indicator, a analog PID control, a digital switch control point ( like a motor MCC with several in- and out-puts )
only by address and numbers, like:
address controller IP, point type, point type index; setpoint number 1 start 2 stop... )
and also the alarming, communicate numbers and on the visualisation PC the number is translated to a TEXT like
0 OK, 1 low alarm, 2 high alarm, 3 high high alarm, or 5 timeout alarm, or for my motor: 11 MCC NOT READY ..
back in my uPCS project i design arrays in python for each point type,
so i can make a process graphic like that
while the arduino only send CSV lines with numbers.
i do a random test generator in a arduino and the RPI should feed that to the mongo database.
i use last AINS program only change to db.collection("MALM").insert(serialData);
check with mongo arduino
> db.MALM.find()
now make a new webserver_actual data page:
and first only as list
npm install
npm start
just for the list it went well, but for further details i was not lucky,
and i new already i beat a dead dog "MONK", i should play "MONGOOSE"
so i copy the webserver project and start new:
now the big question is how to do it ( get records from collections ) in index.js
i got lost, mongoose talk Schema and model
until i found this
var Tank = mongoose.model('Tank', schema);
The first argument is the singular name of the collection your model is for. Mongoose automatically looks for the plural version of your model name. Thus, for the example above, the model Tank is for the tanks collection in the database.
as this still not help me i found the trick to specify the collection name as 3rd argument of the model
but now i want a actual alarm page, what would require a auto update,
i do HTML style, but this require that the motoralarm.jade not use the layout.jade,
only then i can do meta insert there:
doctype html
html
head
title #{title} meta( http-equiv='refresh', content='10')
link(rel='stylesheet', href='/stylesheets/style.css')
body
make a list for the alarmtexts: cat alarmrecord.sh
mongo arduino --eval "db.ALARMS.insert({ 'name' : 'typM01', 'alarmnum':[1,2,3,4,5,6,7,8,9], 'alarmtxt':['a1','a2','a3','a4','a5','a6','a7','a8','a9'] } )"
looks in mongo like that: mongo arduino
> db.ALARMS.find()
{ "_id" : ObjectId("57209af1e940b3191f602243"), "name" : "typM01", "alarmnum" : [ 1, 2, 3, 4, 5, 6, 7, 8, 9 ], "alarmtxt" : [ "a1", "a2", "a3", "a4", "a5", "a6", "a7", "a8", "a9"] }
>
now, before i can even dream of making a VIEW what combines the actual motor alarm number with this above table to retrieve the alarm text,
i need to learn to deal with that arrays,
so i just make a plane alarm text page first:
it took me some time to get that: in
and in
that last loop looks ugly, but that is the only way i found what worked,
i speak not mongoosy but i understand that i just created that problem by my record structure,
so better away from arrays in records.
i start again with a cleaned up table and a easy definition, above i leave in the BLOG just if i run into that kind of thing again. mongo arduino
>show collections
>db.ALARMS.drop()
but now, how to create that view or how it is called in mogoose, not joins, population
the alarm comes with "time" and "alarmnum"
and i need the "alarmtxt" for that "alarmnum" from the table ALARMS
__________________________________________________________
i got distracted by that arduino MCC code / what in return got me a RPI3 sponsored!
( and, you might see in a later BLOG also a arduino M0 board, what i now understand as a .ORG and not .CC arduino board ? and must use .ORG IDE? )
now i first needed to setup that RPI3 and run in bad power problems, the old "mobile" charger worked good on RPI2, but if i need WIFI and arduino...i use a powered USB hub.
Now, possibly because WIFI on board, that charger was too small and caused big problems,
as i work only headless i could not see any color warning on screen.
just that it give me the impression that it boots very slow, i often loose putty connection,
and xrdp not work at all. still i was able to set it up correctly,
so from my weekend workplace i used a acer 5V 2A charger ( from my new acer 7" tablet phone ) and was online via neighbor WIFI, could connect arduino leonardo..
here i am now, try to get that "MEAN project" status back i have on the RPI2.
with some surprises: node new version install:
wget https://nodejs.org/dist/latest/node-v6.1.0-linux-armv7l.tar.gz
pi@RPI3:~ $ node -v v6.1.0
pi@RPI3:~ $ npm -v 3.8.6
pi@RPI3:~ $
//______________________________________________________
and on my RPI2 today install all again to see
2016-05-10-raspbian-jessie.img
hidden treasures ? like the GEANY code editor...
i took off for some time, travel old germany to finish private stuff for good,
and heard a interesting story what made me dig again here.
YAML a CSS framework see here
There are several themes for current CMS like WP made with yaml css framework.
i think this yaml.de
"Yet Another Multicolumn Layout - continuously developed since 2005 "
is NOT connected with this
"YAML is a human-readable data serialization language" "YAML Ain't Markup Language"
where i found that little page: grunt yaml angular to "Grunt task for converting yaml to angular constants." npm i grunt-angular-yaml
lets see what happens?
well, looks like a very tricky thing for specialists, could be useful later..
how the ?old? yaml.org is connected i not understand but its ref sheet is good, see more here.
for use in NODE see here.
While the first, the CSS framework could be useful for a new web page setup, the second "data" structure ( see also XML, JSON, back to CSV ... ) looks clear... to me but not required here and now.
back to ?what? yes Angular, i try to work with this tutorial
and want to combine this tutorial NAV MENU with the YAML layout.
while for a ?file editor? this Menu style looks good.
