Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Robo-Tank Web App - Development
#1
Well I thought it was time to start a thread showing off what I've been working on. I actually started this over a year ago, my original plan was to have it working with my old controller. Overall I've only put a few solid months of work into it though and much of the time spent figuring out a structure I could work with. I started with python but dropped that as I have no previous experience and was having troubles figuring some things out, being a fairly complex system I thought in the end it was best to go with what I know.

You can view the system as two programs, one is the backend which runs all operations for the controller, the other is the frontend which is the webpage used to interact with the controller. 

The backend runs on an Apache server, MySQL is used for the database and PHP is used to update the database from the frontend. I'm using the wiringPi library for the Raspberry Pi pins but managing my own I2C.

https://www.apache.org/
https://www.mysql.com/
https://www.php.net/
http://wiringpi.com/

The frontend is JavaScript, JQuery and CSS. 

https://www.javascript.com/
https://jquery.com/
https://www.w3.org/Style/CSS/Overview.en.html

Running on a Raspberry Pi 3 the CPU runs at about 3-4% and memory is between 150-200mb so there's plenty of resources still available. MySQL is the biggest memory hog and Apache next. I really like MySQL so I think it's worth the extra memory.

The backend is a c++ program, everything is done within. For Arduino folks you can imagine the c++ program is an Arduino sketch and runs in a loop forever however because the Pi has 4 cores the program is running 3 threads, might add a 4th later, which you can think of as 3 Arduinos programs running at once. I went this route because reading DS18B20's causes a 750ms delay in the program every read and unlike an Arduino there's no getting around it, at least I couldn't. I like to read temp sensors every second so one thread handles these and other probes like pH. The 2nd thread is strictly used to communicate from frontend to backed and the 3rd is for the rest of the program.

I'm using a server socket to communicate with backend and frontend. When you press a button on the website it instantly sends the command to the c++ program so it can take action, everything is done in near real time. For some reason communicating to the frontend is more difficult that front to back so I still need to work on this but it works no problems, I just know I'm not doing things right here so that's for another day. For the last 6 months or so I've been running it full time and it's never crashed so feeling pretty good about it. With that said it's not 6 months straight, I think the longest period was about 3-4 weeks.

I haven't set a release date just yet but it shouldn't be much longer, there's a few things I want to add first.

Ok, now for the fun stuff and remember this is just version 1, so much more to come...

Dashboard

This is the dashboard for now, I have other ideas but started with widgets. Once I get the controller fully functioning I can spend more time on UI. As you can see things are sorted into widgets, you can drag and drop the location of the widgets and resize them. Location and size are saved to database so they remain as set. The system is setup for profiles later so you could set a dashboard for desktop and one for mobile etc... The gear in top right corner is for quick access to settings related to what's in widget and some you can sort.

In a later image you can see a side menu, currently it shifts widgets to the right but I'll be overlapping it as it's only a temporary menu to toggle widgets on/off. The top bar will be quick access buttons and system info.

[Image: dashboard3-optimized.PNG]


DS18B20 Temperature Sensors

So I had fun with these and spent a lot of time. My goal with the program is to be the easiest to use (yet packed with features) so I've added an auto detect system. If you plug in a DS18B20 sensor the system will see a new sensor and check if it's already in the database, if it is the sensor will magically appear in the widget within a few seconds of being plugged in and it will have the last known min/max and custom name. If the sensor isn't in the database it will automatically be added and the backend will immediately start logging chart data on the 15 minute interval. If you have the webpage open when you add the new sensor a popup will appear in a few seconds like image below. It's asking for a custom name. After you enter that the sensor will appear in the widget. If you don't have the webpage open when you plug in a new sensor the popup will appear when you open the webpage. If you plugged in say 3 sensors without the webpage open it will cycle through the 3 sensors for a custom name. Obviously that's not recommended unless you know the sensor ID it display but odds of that are very low, I think I memorized mine by now. :) So in the end you wouldn't know what sensor is what. 

When you unplug a sensor it will simply disappear from the widget and the backend will stop writing chart data. Plug back in and it continues like it was never missing. 

On the gauge the min/max is show, that's updated every second with the new incoming temperature and the gauge slowly moves up/down as temperature changes. You can click on the min or max temperature and it'll popup a box like below to verify a reset of the min or max. If you increase the widget size the gauges get bigger, you can see image below of example. 

Note: Background should be greyed out like next image and will be.

[Image: ds18b20_new-optimized.PNG]


If you no longer use a sensor and don't need the chart data you can go in the settings and delete a sensor from the database which will wipe out all data. Eventually you'll also be able to remove select data out of charts easily. 

The table below is a JQuery library, DataTables. I used it to get started quick and it has some nice features. You can sort by clicking any column header and the search is instant with the table updating. 

https://datatables.net/

[Image: ds18b20-optimized.PNG]


Light Modes

You can run up to 50 light modes, can be more though. When you add a mode it appears in the widget list where you can set the slider values. The sliders automatically show based on number of ports you have assigned. As you move the slider the light will respond instantly, once you release it the value will be saved. You can also type in a value from 0 - 4095, later will be an option to show 0-100% instead. When you hover an icon a text appears telling you what the button is.

The checkmarks are obvious, they indicate the days the mode is set for, the checkbox with an orange icon is the next schedule and the green checkbox is the current light mode. 

The green toggle switch allows you to disable the light mode. If you disable "morn" based on image below the orange checkbox will move to "test" mode as that's now the next schedule.

The pencil is to edit the light mode, it pops up the same screen used to add the mode.

The X deletes the light mode.

The up and down icons move the position of the light mode. Note the backend doesn't care what order you put these in, it takes all modes and sorts them based on time / day so you can have day before morn and it wouldn't make a difference. It's only so you can view order you prefer.

You expand a light mode by clicking on it, the sliders will appear for that mode and the lights will automatically switch to those settings. When you toggle the mode closed the light will resume back to the current light mode settings. 

There are no temporary modes yet but technically you can create a light mode with no days selected so it never runs and when you want to view just toggle open and lights will change. When finished close mode and it'll go back to schedule.

The current mode is just something I was getting carried away with, in the end I want visual effects for everything, the beams on the tank will grow/shrink based on light and tank grows based on channels assigned in system. I would love to have an automated tank running with the lights filling it. Also have plans to have a sun moving across the sky, user can switch view they like. So much fun to be had but I need to stay focused. :)

Later I plan for more flexibility with the scheduling but this is a good start.

[Image: lights2-optimized.PNG]

[Image: lights3-optimized.PNG]

[Image: lights-optimized.PNG]


pH and other I2C devices

Like the temperature sensors the I2C also has an auto detect system however it only runs every 10 seconds. Currently I have it working for pH circuits and PCA9685 chips. When a new device is detect on the BUS a popup will appear showing the address and asking what you plugged in. If you select pH it simply asks for a custom name like the temp sensors. Everything else about the system is the same.

If you select the PCA9685 driver it'll ask for a custom name and then allow you to define what the ports will be, 2nd image under construction but works. For function you can select AC Outlet, DC Port, Dosing Pump or Light. When done the widgets will update based on what you do there. If you set 2 pins to "Light" for example two more light sliders would appear on the image above. There's another screen where you can manage all ports and change assignments but overtime it'll all be Q&A type wizards.

[Image: new_I2C_device-optimized.PNG]


If you mouser hover the small icon in table it will enlarge so you can actually see it. ;) The screen to select icon is below.

[Image: new_I2C_device2-optimized.PNG]


Charting and Logs

The charts are being handled by dygraphs, I considered writing my own but hard to match what this does. It's disappointing it's not setup to read from MySQL, instead it uses .csv files so later I'll be working on a solution there. Currently sensor data is being written to both a csv and the database every 15 minutes. I filled up a CSV file with 3 years of data and it only took a few seconds to load so it performs well. 

At the top of widget all sensors found in the database will appear as a checkbox and the user can choose which to view in the widget. Under the chart is a scroll bar, some is shaded, only the non-shaded is being viewed. I have it set to 7 days but a user will be able to change that. You can increase the view by moving one slider or you can move scroller across the whole data set in a 7 days view or whatever you choose. And of course when you hover over line in graph it shows value and timestamp.

One nice feature is you can overlay sensor data on a single chart to compare data sets however a single CSV file needs to be used so it's not useable at this point as there would be a 100 csv file combinations needed. The plan is later I will make a custom chart creator, this is why data is being saved to database as well. A user will be able to choose any chart sets they want to view, a start timeline and the option to keep chart updating or one time view. Then in the background a CSV file will be created that dygraphs accepts and if the user selects to update the system will then continue to save data to the csv file so it stays updated. On the top of the chart will be checkboxes for all data sets you assigned to chart so you can toggle on/off as you like. I've already tested most of this but still to many other things to do.

There are no logs yet but everything will be logged. Using DataTables it won't be too hard to get added and the logs can be sorted and searched efficiently. Later I will add ability to change chart colors etc... 

https://dygraphs.com/

[Image: graphs-optimized.PNG]


AC Outlets / DC Ports

These will basically have the same feature set, currently you can only turn on/off manually so not much use but schedules and water sensors are a coming. 

I'm using the same outlet icons I used for the old controller, an Aussie friend, Iain, made the images for me so I'm very thankful for them, there are 60 in total. As you can see the corners are black as that was needed for the old controller so yeah I have a lot of images to reassemble. :( Thankfully they can be transparent now.

[Image: icons-optimized.PNG]


Manual Port Management

This table shows all available pins or ports you have to work with. With the Robo-Tank controller you would have 39 available. You can assign them as the following to start. Not all ports will be available for all types but only what can be assigned will appear so there's no confusion. At the top of the picture is a dropdown box saying "Raspberry Pi" so only Pi pins are being shown, I'm changing that so all pins are in same table. With the search if you type "Pi" you'll only get Pi pins. This is only temporary anyways, at least it won't be needed later.
  • AC Outlet
  • DC Port
  • pH Probe
  • DHT22 Temperature / Humidity
  • DS18B20 Temperature
  • Non-Contact Sensor
  • Optical Sensor
  • Float Switch
  • Other Water Sensor
  • Flow Meter
  • Light
  • Dosing Pump

[Image: pins_available-optimized.PNG]


When you press "edit" button you get this screen.

[Image: pins_available2-optimized.PNG]



Step by Step Wizards

The UI will be full of wizards and illustrations guiding users through different tasks. You can see an example, I put this together for visual purposes a few months ago, "Add a probe" will be gone as that's automated now. But this is the idea.

Illustrations will be controller and different sensors / equipment showing users how to connect, like they say a picture is worth a 1000's words, for me 5000's words. I'm hoping this can eliminate user manuals as those are no fun for anyone, you get lost in pages of irrelevant info and I always have to update.

[Image: wizards-optimized.PNG]


First Run Setup Wizard

To keep things simple I added a setup wizard that runs once. This will go through all the basic setup parameters required in a friendly manner. I still have illustrations to add but this is the current guide. As you can see in the first image the dashboard is hidden and only the popup visible. When setup is finished it will appear.


[Image: setup1-optimized.PNG]


[Image: setup2-optimized.PNG]


[Image: setup3-optimized.PNG]


[Image: setup4-optimized.PNG]


[Image: setup5-optimized.PNG]


Here it will scan for I2C devices and popup the wizard shown earlier. There will be a question before this one I forgot, it will ask if you use Robo-Tank controller (highly recommend) :) or other. If Robo-Tank is selected the pH and PCA9685 will be added behind the scenes, it'll only scan for extras that might be plugged in.

[Image: setup6-optimized.PNG]


[Image: setup7-optimized.PNG]


That's it for now, here's a few more images of different dashboard layouts.

[Image: dashboard2-optimized.PNG]


[Image: dashboard-optimized.png]


[Image: dashboard4-optimized.PNG]
[-] The following 2 users Like Rob F's post:
  • JFReyes, Weasel1960
Reply to top
#2
K05164 Hi ! I can’t wait to try this new program ! It’s really a good job, congratulation.
[-] The following 1 user Likes tutuss21's post:
  • Rob F
Reply to top
#3
Congratulations Rob! I look forward to trying it out...
José

Please post questions in the forum (not via PM) so that everyone can benefit from and contribute to finding solutions. Thank you.
[-] The following 1 user Likes JFReyes's post:
  • Rob F
Reply to top
#4
Very nice and clear information and same for the menu's. What does Fauci think of it? Just kidding....
Rob, this is and will be cool.

J
[-] The following 1 user Likes Jerry W's post:
  • Rob F
Reply to top
#5
Whaouu! I also look forward to try it.
[-] The following 1 user Likes Dom59's post:
  • Rob F
Reply to top
#6
Thanks guys, looking forward to getting it released.
Reply to top
#7
Wink 
Looking forward to use this version 

Will order another kit soon also K05108
[-] The following 1 user Likes fred73's post:
  • Rob F
Reply to top
#8
wow that looks great and has so many opportunities to be used for. and looks straight forwarded to use. you did a good job on it Rob.
[-] The following 1 user Likes rott's post:
  • Rob F
Reply to top
#9
Rob, seeing all that euphoria K05164 K0507  give It up is forbidden  K05103 K05163
[-] The following 1 user Likes Dom59's post:
  • Rob F
Reply to top
#10
(04-23-2021, 07:51 AM)rott Wrote: wow that looks great and has so many opportunities to be used for. and looks straight forwarded to use. you did a good job on it Rob.

Thanks rott, still lots to go but I still have many years left ahead so lets see what happens. :)

(04-23-2021, 10:31 AM)Dom59 Wrote: Rob, seeing all that euphoria K05164 K0507  give It up is forbidden  K05103 K05163

Yes I will never give up, I have a vision lol.
[-] The following 1 user Likes Rob F's post:
  • Dom59
Reply to top
#11
Hi Rob, this is an amazing job.

It will be possible for you to provide me with files for editing the language interface so that he can translate all the texts as I once solved it in robotank.
Reply to top
#12
(04-25-2021, 02:03 PM)loksik.lubos Wrote: Hi Rob, this is an amazing job.

It will be possible for you to provide me with files for editing the language interface so that he can translate all the texts as I once solved it in robotank.

Thanks Loksik, I'm not quite ready to release it yet but when I do you can make changes. I expect this will be a lot easier to translate than the old controller.
Reply to top
#13
Time for a little update. I'm making some gains but it will still be another month at least, I've decided I need more done before releasing but I'm quite happy with how it's coming along, everything has been working great.

I got through quite a few more minor details for what was already working and added an .htaccess file to finally sort out caching different file types, there was a problem with icons sometimes not updating because it was reading cached files instead of from the server so that little mystery is solved.

The pins available table I posted above has been changed so all available pins are in the same table. I added a little color so it's easier to spot which pins are used vs available. Again this is just temporary, later it'll be converted to step by step guides but this will remain non the less. This auto updates when new I2C devices are detected.

[Image: pins_available3-optimized.PNG]


Schedules

The new addition is the scheduling system, I haven't added it to the backend yet as I want to get the custom rules and custom modes running at the same time as they all work together in some form. 

I like the DataTables jQuery library I was using, the above table is it however it looks horrible on mobile and doesn't fit the screen, you have to scroll horizontally. That's why making the schedules took longer than I planned as I decided to make my own sortable table with customizable pagination which was a bit of a challenge, at some point I'm going to see if I can add an instant Ajax search function like DataTables as that is a sweet feature and then I can get ride of it. For large tables that search will be needed. 

I'm not 100% happy with the look but will tweak over time. On the right side of the top header you can see 7 icons, those are to filter what you're viewing and your selection is saved. There's a setting to change this to a dropdown box as well, a picture following shows it. You can filter by All schedules, AC, DC, Dosing, Feeding, Custom Rules or Custom Modes. You can click on any column to sort ascending or descending. In the image below it's sorted by schedules on Mondays. The up/down grey icon turns aqua and icon changes to a ASC or DESC icon, 3 icons total.

You can change how many schedules show per page but I've yet to place that setting somewhere, it is saved as well.

This view is on desktop, when the screen size shrinks as in different devices being used columns will auto hide so it always fits on the screen. The last couple images is a screen that's 380 pixels wide.

Like the light modes you can enable/disable a schedule, delete a schedule or edit basic parameters using the icons on the right side.

Forgive the poor schedule examples, I quickly added and duplicated data to get the table working.

[Image: schedules-optimized.PNG]

If you click on a row in the table it will expand and spell out what the schedule will do and eventually show the next schedule upcoming. The 2nd line saying "If maintenance is running schedule will run" will be changing as I've decided to go a slightly different direction with feeding and maintenance modes. I'll explain that later.

[Image: schedules2-optimized.PNG]

This is filtered to show only AC outlets, you can see the dropdown box instead of icons on this picture. Yes looks like that dropdown needs to be sorted.  K0505  K05163

[Image: schedules4-optimized.PNG]

The picture below you can see the available features on the scheduling side. All schedules except for lights use the same millisecond system. Basically all schedules in the system backend are converted to millis and then goes through each as time goes by.

You can select if schedule is on or off and what days or X number of days it will run. If you only wanted a schedule to run every 30 days you can tick the "repeat" box, the days will disappear and enter how many days between schedules you want. I'll also be adding the option for a one time schedule that deletes it's self after running. If you wanted someone scheduled every Monday and every 30 days you can create two schedules, there's no limit on how many you can set for any piece of equipment.

The option at the bottom will be moved to the custom modes system which can be used for maintenance, feeding or whatever you desire. 

[Image: schedules3-optimized.PNG]

The next two pictures are at 380 pixels wide, same as my phone. 

[Image: schedules6-optimized.PNG]


[Image: schedules5-optimized.PNG]


Custom Rules

Currently I'm throwing together something to get the custom rules running. Custom rules are what you use to control various equipment using any available parameter. The main widget page will have a table showing all custom rules similar to schedules but I haven't made it yet. 

When you create a custom rule you first select how the rule will be triggered, it can be via schedule, sensor or manually. 

[Image: customRules-optimized.PNG]


If you select sensor the next page will show all sensors in system, water, temp, various probes, flow meters etc, anything that outputs info. Later I'll expand it to respond to AC/DC port statuses. The names under the icons is the custom name you use for them. I'll do better with the icons later.

[Image: customRules2-optimized.PNG]


Once you select the sensor, in this case a temperature sensor, it'll ask to set the level. The initial value shown will be whatever the sensor currently reads, then you can adjust the whole or decimal number using the buttons and select if <= or >=. This will be the same screen for pH type probes, humidity etc. 

[Image: customRules3-optimized.PNG]

If you selected an optical sensor, non-contact sensor or float switch the option would be "Touching Water" or "Not Touching Water". 

[Image: customRules4-optimized.PNG]

That's as far as I got. K05106  

After this step it will ask what equipment you want to control, everything connected will be available and you can select multiple pieces of equipment including ability to run a dosing pump that's been setup to do it's thing.
Next based on equipment selected it will ask a few more very important questions but you need to stay tuned for those.  K0519
[-] The following 3 users Like Rob F's post:
  • albinochicken89, JFReyes, Weasel1960
Reply to top
#14
Hey Rob! Wanted to check in and see how this is going? I’m excited to use it as I think it’ll be much better running macros and such. Plus I like that it has a UI that’s easy to read.
Reply to top
#15
(06-13-2021, 12:30 PM)albinochicken89 Wrote: Hey Rob! Wanted to check in and see how this is going? I’m excited to use it as I think it’ll be much better running macros and such. Plus I like that it has a UI that’s easy to read.

Hey albinochicken89, it's coming along nicely, been working on it steady since the last post. Made quite a few improvements and lose ends tied that can't be seen and also have the scheduling system complete turning outlets on/off. Now I'm working on getting the dosing pump system running. The backend isn't too difficult, it's the frontend that takes all the time, I'm rather picky and mess with the smallest detail way to long when it's not even a big deal haha. 

I haven't decided when I'll do a release, there's still things I want to complete first as my version of macros, call custom rules, ties everything together so without that there's no automation. Plus first impression is everything so I don't want to many UI quirks.

After dosing pumps I'm going to add flow meters which shouldn't take long, then I need calibration for pH and finally custom rules which will take work but I know what I want at least. At that point it should be basically ready to cover everything on the controller.

This is the dosing pump monitor, I can't say I like it too much, was suppose to look like my old controller but not so. Either way it'll work for now. I'll be doing the setting screens this week. This will auto update as they run. Eventually all UI colors can be custom but I will be adding it right away for the reservoirs, each can have it's own color.

To give an idea on how it works, if you assign a port as a dosing pump it will show up in the dosing pump widget, if you changed the port to a light it would prompt and be removed from dosing pump and show up as an LED channel, the system will always clean up after itself.

So yeah I'm getting excited to have it come together, really happy how it's coming along just a lot of work.   K05178

[Image: dosing_pumps.PNG]
[-] The following 3 users Like Rob F's post:
  • albinochicken89, JFReyes, Weasel1960
Reply to top
#16
this is looking good can not wait to use it
Reply to top
#17
Hey rott, every day is one day closer. :) Working on it tonight.
Reply to top
#18
Hi Rob just think 6 months from now you will be using R Pi's for everything :)
[-] The following 1 user Likes rott's post:
  • Rob F
Reply to top
#19
Hi Rob. Thanks for the link you PMd me in R2R. This is totally awesome, like your older version videos on YouTube this looks very user friendly...I like things that are simple and a little bit (a lot actually) of magic in the background never hurts.??

Well done and look forward to future updates.
[-] The following 1 user Likes Weasel1960's post:
  • Rob F
Reply to top
#20
Thanks Weasel1960, I still like the older better but this will get there.

Well a little update. The dosing pumps are basically done still a little clean up to do and add the calibration. 

First I had to change that dosing vessel, I think this one looks much better.

[Image: dosing_pumps2.PNG]


This is the settings screen, you get here pressing the gear in header. I'll be adding a pencil icon next to the custom name so it's clear it can be edited.

[Image: dosing_pumps3.PNG]


In the pic below you can see a color picker, if you click on the color in the reservoir it will open and you can change to any color.

For now I'm using jscolor color picker, it works very well. I'm thinking maybe throughout the program I will use that with a double click. Anything that can change color you would double click and it would appear. Just an idea but has downside of people needing to discover that or read in a manual which I don't like. 

https://jscolor.com/

[Image: dosing_pumps5.PNG]


This is the "more settings" screen. I don't have the calibration screen yet. If reservoir need to go from 2500ml to 250ml don't worry it's not 2250 button presses, the longer you hold the button the quicker the number changes, same goes for dose amount. I was trying to also have it so if I pressed and held the number it would turn to an input so user could also type in value but that was being stubborn so I put it off for now.  K05163

The pump rate is calculated during calibration.

[Image: dosing_pumps4.PNG]
[-] The following 2 users Like Rob F's post:
  • albinochicken89, Weasel1960
Reply to top


Possibly Related Threads…
Thread Author Replies Views Last Post
  How to Install Robo-Tank App Rob F 8 1,555 02-04-2024, 06:44 PM
Last Post: Rob F

Forum Jump:

Current time: 03-28-2024, 05:45 PM