04-24-2022, 07:05 AM
I am sorry this happened, definitely not what I want to hear but now I think I understand. As for your changes I haven’t ignored them, I appreciate all feedback. I thought I mentioned I wasn’t going to add the dosing pump changes as I was going to redo it all which I’m currently in the process of. I’m adding your suggestions such as schedules dictate how much is dosed etc. I’m also always improving existing code and adding fail safes, this is why some updates don’t look impressive but changes and improvements are always being done. This is why I added the pca9685 verification and logging, to improve reliability, I felt that was more important then dosing features at the time. Unfortunately I’m also limited on time I can spend on this, that’s why I haven’t started a thread yet on future features as there’s still to many other things that need addressing and it would feel like I’m ignoring them.
As for that variable size not being large enough, that was only an issue if well the calculated number was too large. In all my testing I never thought to have such a large dose or whatever caused the large number so I never came across the issue so it wasn’t one, it’s impossible for me to configure test setups for every possible scenario. That’s why I need beta testers, as you’re a software developer you should understand that it takes time and small details can be overlooked sometimes. I thank-you for discovering the issue and now we need to find the next cause there will be one. This is why it’s important to test everything after adding a schedule, custom rule or whatever to verify it functions as expected.
Putting aside that this caused you harm, the pca9685 verification did exactly what I hoped and sniffed out an issue. Your correct that as is in 6.5 it’s possible to run forever which would be the same effect as the program freezing and shortly after I released it I shook my head that I didn’t add an escape method, this is something I’m always aware of and make sure it can’t happen, I guess I rushed it out without realizing it but it’s there now. The same idea as yours, just count the retries and exit if too many.
Your fix won’t do anything though, the reason it went in an infinite loop is because it lost all communications with the pca9685 so aborting after 3 tries and then essentially trying a 4th time to set pin off will do nothing as it can’t communicate, may as well just make the loop 4 tries and abort. One of the beauties of the verification is we know every time this happens, I can foresee it happening rarely but only to the point where the 2nd maybe 3rd command sent is valid. If it loses all communication or happens often something needs addressing with the hardware. If the I2C bus hangs, which sounds like it is, the only fix is a reset and that’s easy now and something I’ll add. After it tries say 10 times it will then create a log entry and alert saying the controller was restarted due to pca9685 timeout and then reboot the controller. When the Pi restarts the pca9685 will be reinitialized and all outputs set to off. This might be the safest and should always get the pca9685 reset within 20-30 seconds (Pi boot time) of failure before it sorts itself out. As it’ll be clearly logged plus alert one will know when it happens and the problem can be looked into and fixed so the validation is invaluable, without it one is guessing, now we know so I double down on being very excited about this feature and even more so. If this was in place a week ago with a reboot I think what happened would have been avoided minus the extra dosing due to long start up time and we still would have known it happened. This is something I hope to improve later. Currently the service file doesn’t start the program until a network connection is verified as its needed for NTP time. Now that an RTC is available I don’t see why it needs to wait, hoping to have the program up and running in under 5 seconds at some point.
So yeah now I think the problems you’ve had with dosing was never related to code minus the variable oversight, it’s always done what it should. As you’ve never had issues with doses starting or other things that means scheduling system works (logging will help verify) and now that the pca9685 verification has told us the I2C bus locked we know it’s hardware. No software is able to shut of a pin if it can’t communicate.
Looking at your order I see you have an external pca9685, do you only have that connected or other I2C devices as well? How long are the cables going to the external device/s? My gut thinks it may be that causing instability on the I2C bus, it doesn’t take much. This is why for your kH tester I didn’t like to hear I2C, it can cause a bad day and ultimately I do need to lose it for expansions like the pca9685.
(04-24-2022, 03:17 AM)gandalf Wrote: You said the the dosing system is running properly from years, but the fix in v6.4 for the variable overflow that lead to an unlimited dose is mine....
As for that variable size not being large enough, that was only an issue if well the calculated number was too large. In all my testing I never thought to have such a large dose or whatever caused the large number so I never came across the issue so it wasn’t one, it’s impossible for me to configure test setups for every possible scenario. That’s why I need beta testers, as you’re a software developer you should understand that it takes time and small details can be overlooked sometimes. I thank-you for discovering the issue and now we need to find the next cause there will be one. This is why it’s important to test everything after adding a schedule, custom rule or whatever to verify it functions as expected.
Putting aside that this caused you harm, the pca9685 verification did exactly what I hoped and sniffed out an issue. Your correct that as is in 6.5 it’s possible to run forever which would be the same effect as the program freezing and shortly after I released it I shook my head that I didn’t add an escape method, this is something I’m always aware of and make sure it can’t happen, I guess I rushed it out without realizing it but it’s there now. The same idea as yours, just count the retries and exit if too many.
Your fix won’t do anything though, the reason it went in an infinite loop is because it lost all communications with the pca9685 so aborting after 3 tries and then essentially trying a 4th time to set pin off will do nothing as it can’t communicate, may as well just make the loop 4 tries and abort. One of the beauties of the verification is we know every time this happens, I can foresee it happening rarely but only to the point where the 2nd maybe 3rd command sent is valid. If it loses all communication or happens often something needs addressing with the hardware. If the I2C bus hangs, which sounds like it is, the only fix is a reset and that’s easy now and something I’ll add. After it tries say 10 times it will then create a log entry and alert saying the controller was restarted due to pca9685 timeout and then reboot the controller. When the Pi restarts the pca9685 will be reinitialized and all outputs set to off. This might be the safest and should always get the pca9685 reset within 20-30 seconds (Pi boot time) of failure before it sorts itself out. As it’ll be clearly logged plus alert one will know when it happens and the problem can be looked into and fixed so the validation is invaluable, without it one is guessing, now we know so I double down on being very excited about this feature and even more so. If this was in place a week ago with a reboot I think what happened would have been avoided minus the extra dosing due to long start up time and we still would have known it happened. This is something I hope to improve later. Currently the service file doesn’t start the program until a network connection is verified as its needed for NTP time. Now that an RTC is available I don’t see why it needs to wait, hoping to have the program up and running in under 5 seconds at some point.
So yeah now I think the problems you’ve had with dosing was never related to code minus the variable oversight, it’s always done what it should. As you’ve never had issues with doses starting or other things that means scheduling system works (logging will help verify) and now that the pca9685 verification has told us the I2C bus locked we know it’s hardware. No software is able to shut of a pin if it can’t communicate.
Looking at your order I see you have an external pca9685, do you only have that connected or other I2C devices as well? How long are the cables going to the external device/s? My gut thinks it may be that causing instability on the I2C bus, it doesn’t take much. This is why for your kH tester I didn’t like to hear I2C, it can cause a bad day and ultimately I do need to lose it for expansions like the pca9685.
