Hello, I'm happy to say I have an update for the web app, sorry it's been so long. Quite a bit has been done and I'm feeling pretty good about it's future.
Thanks to everyone who has tested the app and given feedback, it's all been very helpful. Any more you have would be appreciated. The next post will go over the update and fresh install process.
Here's what's been done.
* Database connection pool - This is big. Due to the threads there was an issue with MySQL database connections which rarely occurred but when it did it would crash the program. It would only happen when saving custom rules or during startup when the DS18B20 sensors loaded up. With custom rules it still updated the database and the program would restart in 3 seconds due to the Pi service running so it wasn't causing huge issues but obviously a problem. I made some changes in v6.6 which helped but that's it. Finally I got it solved using a MySQL connection pool, it was happening due to a variable being shared across the threads. I could see it happen as I run the program in the SSH terminal, when it would start it would sometimes crash after loading the DS18B20 sensors. About 6 months ago I made this change and it hasn't happened since, I'm 100% confident it's fixed and feel great about it.
* Emails - These are now sent on a separate thread and don't interfere with the program in any way. Previously they would pause the main loop briefly which wasn't ideal, now everything runs uninterrupted, it's real time 100% of the time.
* Charts - I'm happy to say all the chart data is now coming from the database, previously it was CSV files. I had the foresight on this, from day 1 the data was being saved to the database so you shouldn't notice a difference but there is. This is great because now queries can be created to pull data from multiple sources and placed on the same chart. Possibly the next release you'll be able to make custom charts. Below is the chart for the DHT22, it shows temp and humidity and the next one is the dosing pump. You can see in that chart with the mouse hovering it dosed 7.14ml and took 4.29 seconds. The seconds in the chart is measured for each dose, not calculated. It's very accurate and shows how fast the system runs, each dose within 100th of a second.
* DS18B20 Temperature Sensors - A fair bit was done with these, finally there's a settings page.
You can edit the name, set how many days show up in the charts keeping data usage down (data remains in database) and you can enable a "Malfunction" email alert. Because the system knows when a temp sensor isn't communicating you can have it send you an alert if it stops working, unplugging it will also set off the alert. You can also delete all the charting data, sorry the heater and chiller buttons do nothing yet.
You can also calibrate the sensors using an ice bath.
It's automated, just start it and wait.
I revisited the dosing pumps and updated the scheduling system, now it's a different system then other scheduling. I think I'm going this route for other schedule types, this way it can be customized for equipment types.
When you click the "View Details" button you get a list of the schedules, you can see some of them are group schedules. Now group schedules are stored as one schedule in the system saving lots of memory. The schedules within a group are dynamically calculated each time a schedule runs. This took a lot of effort as schedules can end after midnight but I think everything is good. Be sure to monitor things and check the log for the first while.
In the summary below it shows the total amount being dosed for the group.
You can click on any schedule and it will expand with all the details. For now the "Next Schedule" is just a date and time but I will be changing this to say things like today, tomorrow, in 4 days, etc...
Here's the screen to create the schedule. By default it for a single schedule, if you click the "Recurring Schedule" option you can the next image.
Now with "Recurring Schedule" ticked you get a field for the "Last Dose" and a "Run Schedule All Day" option. It can calculate the group based on either dose amount or number of doses in time frame. It'll only allow you to set one of them.
If the math doesn't add up it'll automatically adjust the total dose amount and explain why it did this, here's an example.
You can also access the log for each dosing pump, this just filters the main log which I will be improving soon.
There is now flow meter support, you can connect flow meters like the link below directly to a sensor port. For now you can only change the calibration factor value but at some point there will be a calibration process. You can use the flow rate in custom rules and there's a chart for flow rate. When you add a flow meter it appears on the dashboard with the other sensors, you can see one in the dashboard image above. It displays flow rate, total flow and min/max flow rate.
Amazon.com Flow Meter
There was a few things done with the custom rules, DHT22 was added, you can use either temperature or humidity for the rule or both if you wish. Flow meter flow rate was also added. If you don't have flow meters or water sensors as an example, the titles won't be in the drop down.
You can see a new option for the custom rules, this is something I've gone back and forth on and it finally hit me, just have an option. This only appears if you add a switching delay to any piece of equipment. There's two types of delays, you can set a stability delay for the sensor so the custom rule doesn't trigger unexpectedly and there's a switching delay you can add to each piece of equipment allowing you to add steps. The problem with this delay was what should happen if the sensor resets, should the rule abort or should it run until all the actions complete. When I say sensor resets I mean, if the custom rule was triggered when a sensor wasn't touching water, then it touched water again before the custom rule finished what should happen. I think some scenarios you want it to finish others you don't. Hopefully it doesn't cause to much confusion. Let me know what you think.
In the preview it now states "Custom rule will run until completed!".
Here's the settings menu, a few more buttons. Sorry the heaters aren't setup yet, I was going to do but it's time to release update.
There's a new setting on the System Settings page in the first box. This is "Auto Logout". Previously it was pretty random when you would be logged out, maybe a day maybe 2 days. Quite a few people were annoyed by it along with myself so I think that's solved. When the app is installed the php.ini session timeout is changed to 1 year. In the app you set how many hours you want to stay logged in and when that expires a function will run and kill the session.
That's about it for pictures, here's a summary of all the changes. You can find this in the "Version Log" seen in the image above.
Fixes
Thanks to everyone who has tested the app and given feedback, it's all been very helpful. Any more you have would be appreciated. The next post will go over the update and fresh install process.
Here's what's been done.
* Database connection pool - This is big. Due to the threads there was an issue with MySQL database connections which rarely occurred but when it did it would crash the program. It would only happen when saving custom rules or during startup when the DS18B20 sensors loaded up. With custom rules it still updated the database and the program would restart in 3 seconds due to the Pi service running so it wasn't causing huge issues but obviously a problem. I made some changes in v6.6 which helped but that's it. Finally I got it solved using a MySQL connection pool, it was happening due to a variable being shared across the threads. I could see it happen as I run the program in the SSH terminal, when it would start it would sometimes crash after loading the DS18B20 sensors. About 6 months ago I made this change and it hasn't happened since, I'm 100% confident it's fixed and feel great about it.
* Emails - These are now sent on a separate thread and don't interfere with the program in any way. Previously they would pause the main loop briefly which wasn't ideal, now everything runs uninterrupted, it's real time 100% of the time.
* Charts - I'm happy to say all the chart data is now coming from the database, previously it was CSV files. I had the foresight on this, from day 1 the data was being saved to the database so you shouldn't notice a difference but there is. This is great because now queries can be created to pull data from multiple sources and placed on the same chart. Possibly the next release you'll be able to make custom charts. Below is the chart for the DHT22, it shows temp and humidity and the next one is the dosing pump. You can see in that chart with the mouse hovering it dosed 7.14ml and took 4.29 seconds. The seconds in the chart is measured for each dose, not calculated. It's very accurate and shows how fast the system runs, each dose within 100th of a second.
* DS18B20 Temperature Sensors - A fair bit was done with these, finally there's a settings page.
You can edit the name, set how many days show up in the charts keeping data usage down (data remains in database) and you can enable a "Malfunction" email alert. Because the system knows when a temp sensor isn't communicating you can have it send you an alert if it stops working, unplugging it will also set off the alert. You can also delete all the charting data, sorry the heater and chiller buttons do nothing yet.
You can also calibrate the sensors using an ice bath.
It's automated, just start it and wait.
I revisited the dosing pumps and updated the scheduling system, now it's a different system then other scheduling. I think I'm going this route for other schedule types, this way it can be customized for equipment types.
When you click the "View Details" button you get a list of the schedules, you can see some of them are group schedules. Now group schedules are stored as one schedule in the system saving lots of memory. The schedules within a group are dynamically calculated each time a schedule runs. This took a lot of effort as schedules can end after midnight but I think everything is good. Be sure to monitor things and check the log for the first while.
In the summary below it shows the total amount being dosed for the group.
You can click on any schedule and it will expand with all the details. For now the "Next Schedule" is just a date and time but I will be changing this to say things like today, tomorrow, in 4 days, etc...
Here's the screen to create the schedule. By default it for a single schedule, if you click the "Recurring Schedule" option you can the next image.
Now with "Recurring Schedule" ticked you get a field for the "Last Dose" and a "Run Schedule All Day" option. It can calculate the group based on either dose amount or number of doses in time frame. It'll only allow you to set one of them.
If the math doesn't add up it'll automatically adjust the total dose amount and explain why it did this, here's an example.
You can also access the log for each dosing pump, this just filters the main log which I will be improving soon.
There is now flow meter support, you can connect flow meters like the link below directly to a sensor port. For now you can only change the calibration factor value but at some point there will be a calibration process. You can use the flow rate in custom rules and there's a chart for flow rate. When you add a flow meter it appears on the dashboard with the other sensors, you can see one in the dashboard image above. It displays flow rate, total flow and min/max flow rate.
Amazon.com Flow Meter
There was a few things done with the custom rules, DHT22 was added, you can use either temperature or humidity for the rule or both if you wish. Flow meter flow rate was also added. If you don't have flow meters or water sensors as an example, the titles won't be in the drop down.
You can see a new option for the custom rules, this is something I've gone back and forth on and it finally hit me, just have an option. This only appears if you add a switching delay to any piece of equipment. There's two types of delays, you can set a stability delay for the sensor so the custom rule doesn't trigger unexpectedly and there's a switching delay you can add to each piece of equipment allowing you to add steps. The problem with this delay was what should happen if the sensor resets, should the rule abort or should it run until all the actions complete. When I say sensor resets I mean, if the custom rule was triggered when a sensor wasn't touching water, then it touched water again before the custom rule finished what should happen. I think some scenarios you want it to finish others you don't. Hopefully it doesn't cause to much confusion. Let me know what you think.
In the preview it now states "Custom rule will run until completed!".
Here's the settings menu, a few more buttons. Sorry the heaters aren't setup yet, I was going to do but it's time to release update.
There's a new setting on the System Settings page in the first box. This is "Auto Logout". Previously it was pretty random when you would be logged out, maybe a day maybe 2 days. Quite a few people were annoyed by it along with myself so I think that's solved. When the app is installed the php.ini session timeout is changed to 1 year. In the app you set how many hours you want to stay logged in and when that expires a function will run and kill the session.
That's about it for pictures, here's a summary of all the changes. You can find this in the "Version Log" seen in the image above.
Fixes
- Apache - Fixed various errors showing up in Apache2 error log
- Charts - Chart data was loading when hidden
- Charts - Could not hide pH charts
- Configure Ports - Port was not taken out of manual mode when function was changed, this could prevent it from switching
- Custom Rules - If custom rule had a delay before switching equipment it was possible the custom rule resets before actions are taken
- Dosing Pumps - If reservoir was disabled level remaining would go below 0 milliliters
- Dosing Pumps - Sometimes stirrers would not work, fixed array location errors
- DS18B20 - If a DS18B20 sensor loses communication, the system tracks it and prevents temperature-based custom rules from false triggering
- User Contributed - DS18B20 - If the DS18B20 sensor ID did not contain a letter you could not create a custom rule with it
- Schedules - A schedule could not be edited after adding dose amount to scheduling system
- Schedules - Rounded dosing pump milliliters to 1000ths on schedule list
- Startup - If a pca9685 was not detected the program would not start
- Startup Wizard - Fixed controller version selection during first time run wizard
- System - Removed MySQL database global variable for connections and replaced with connection pool
- Charts - Added DHT22 chart, it displays temperature and humidity
- Charts - Added dosing pump chart, it displays milliliters dosed and number of seconds it ran
- Custom Rules - Added option to abort or complete custom rule if sensor resets
- Dashboard - Added shortcuts on dashboard sensors, click any sensor to go directly to settings
- DHT22 - DHT22 temperature and humidity sensor has been added back
- DS18B20 - Added an optional malfunction alert if DS18B20 temp sensor is unplugged or stops working
- DS18B20 - Added ice bath calibration process
- DS18B20 - Added screen to edit DS18B20 temperature sensor details and set options
- Email - Added button on settings page to send a test email
- Flowmeters - Added flow meter support
- I2C Bus - Added email alert and system reboot if pca9685 verification failed and timed out
- System - Introduced a setting on the System Settings page to configure the auto logout timer, in hours
- Charts - Data is now read from database instead of CSV files
- Charts - Temperature and pH charts are now updated every 8 minutes instead of 5 minutes
- Charts - You can now set the maximum number of days that is loaded in the chart, all data remains in database
- Database - Updated all queries to use PDO to access, added prepared statements for security and added error proofing
- DHT22 - Removed DHT22 from DS18B20 system, now using the pinsAvailable table as they connect to Pi GPIOs
- Email - Emails are now sent using a dedicated thread, the main program and critical operations continue running without interruption
- Updates - Improved update system and added configurable update URL
Last edited:
