Ensure that CrowCam works on DSM 7.x

[tfabris]

Currently the CrowCam code is tested and working on Synology DSM version 6.2.

There is a DSM 7.x version available, and 6.2 will reach end-of-life soon. I received an email that said support for DSM 6.2 will end on October 1st, 2024. I should make sure to upgrade my Synology to version 7 and ensure that CrowCam works on it.

I had an email conversation with Alex Santantonio from https://oldtownhome.com - He's using the CrowCam code to run his OspreyCam on YouTube at https://www.youtube.com/@OldTownHomeBlog/streams - He says that he needed to make some changes to the script to make it compatible with DSM 7.x. I'm trying to get a hold of him again to get some tips from him.

My guess is that his code and my code have diverged enough at this point that a merge would be difficult, but I'd certainly like to see Alex's changes that he made to make it DSM 7.x compatible.

[quog]

One issue:
IsServiceUp in CrowCamHelperFunctions.sh uses synoservicectl which doesn't exist anymore in DSM 7.2
There is
sudo synosystemctl get-active-status pkgctl-SurveillanceStation.service
but it needs sudo which is an issue.
I commented out the service check on line 918 of CrowCam.sh

[quog]

Second issue (I'm debugging as I go) - I had to replace
authWebCall="auth.cgi?api=SYNO.API.Auth&version=1&method=login&account=$username&passwd=$password&session=SurveillanceStation&format=cookie"
with
authWebCall="entry.cgi?api=SYNO.API.Auth&version=3&method=login&account=$username&passwd=$password&session=SurveillanceStation&format=cookie"
in CrowCamHelperFunctions.sh line 1100

[quog]

One other issue, ping is no longer able to run unless you sudo - you get
ping: socket: Operation not permitted
Two solutions:
chmod +s /bin/ping
which I don't like as it will potentially be lost with a DSM upgrade
So I changed line 650 of CrowCam.sh to be
if nslookup $ThisTestSite 8.8.8.8 >/dev/null
as nslookup still works for a user, 8.8.8.8 resolves as dns.google, and nslookup returns appropriately for the if statement. And by having the server parameter, it means internal DNS caching is bypassed.

[quog]

FYI - the stream is at https://www.youtube.com/channel/UCIo6tmAPHtFduP3of7quqOw/live

[tfabris]

@quog - Thank you so very much!

Regarding "IsServiceUp" and "synoservicectl" - I looks like this was originally done as bulletproofing, just to make sure that someone who was running this script truly met the requirements. By which I mean, being on a Synology with Surveillance Station installed and running. I don't think that level of bulletproofing is truly necessary, and since it doesn't work any more, it makes sense to skip it. So indeed I will entirely remove the code which calls synoservicectl.

Regarding the "auth.cgi" versus "entry.cgi" change, and the move from v1 api to v3 api: I'll have to research the difference between those CGIs for the API endpoint. If you have more information about what the difference is between "auth" and "entry", let me know, otherwise I'll have a scrape around the docs about it. Additionally, I'm going to need to figure out why I needed to back down to V1 of the API. I made a note to myself about having had troubles with the V3 API, and I'm going to need to do some experimenting to find out exactly what those troubles were. I should have left myself better reminders, so that I know what to look for.

Regarding the change of "ping" to "nslookup" - This seems like a logical change on the face of it, but I need to understand it in more detail. In particular, I need to be absolutely certain that I'm not getting a cached version of the DNS from my local router when I use nslookup in favor of ping. You say that it should work, and I believe you, but sometimes I've seen other network commands lie to me before and feed me cached data (even when their docs say they shouldn't), and I have to fall back to using actual ping to get real time information. I also don't understand why sudo is going to be required to run ping if the script is running under root. When I set these things up on my synology in my Task Scheduler, the script is already running under root and sudo shouldn't be needed. Are you sure that sudo is really required to run ping, you're not just debugging the script under some lower account? Have a look at https://github.com/tfabris/CrowCam/blob/master/README.md#usage-and-troubleshooting and see if the notes there about running it with sudo and running under root in the task manager are applying to your situation. I don't have V7 installed yet, so I don't know if the task manager works any differently, so you might have more information on this than me.

Thanks again! Let me know if you find anything else.

[quog]

Hi Tony,

Those were the only changes needed to make it work on my client's NAS running DSM 7.2.1. Note: I have the scheduled tasks running as an admin user, but not as root - and that seems to be working (at least so far.) It's only been running for a couple of hours so I will see what happens tomorrow.

And yes, ping requires root or sudo now. The change to using nslookup was primarily because my client often had internet issues, and I didn't want to remove that check altogether. By giving two parameters to nslookup, you are telling it to talk to the DNS server in the second parameter which in my example is outside their network. So unless something like a firewall is proxying DNS, you are pretty much guaranteed to have a network connection if that command works.

I found the new Synology APIs here: https://global.download.synology.com/download/Document/Software/DeveloperGuide/Os/DSM/All/enu/DSM_Login_Web_API_Guide_enu.pdf

Thanks for the great code.

Regards
Damian

[tfabris]

@quog - Understood, thanks so much! Your comments are very helpful. Since my NAS is still running DSM 6.x, I don't have a way to test on DSM 7.x at the moment. My intention is to upgrade my NAS to a newer model which is already running DSM 7.x, and then migrate the data and scripts over to it, but I haven't done that yet.

I've updated the code to remove synoservicectl already. Regarding the other two changes...

Clarifying questions:

• If the scripts had been configured to run from the Synology Task Scheduler as "root" with the "bash" command line (as described in the configuration instructions here), then would the existing "ping" command have worked without modification?

• After you modified the the "WebApiAuth" function to use "entry.cgi" and "version=3" to log in, did the other API commands still work without modification? Examples:

  • CrowCamHelperFunctions.sh
    • "IsStreamRunning"
    • "StartStream"
    • "StopStream"
    • "CreateNewStream"
  • CrowCam.sh
    • Three instances of calls to "entry.cgi?api=SYNO.SurveillanceStation.YoutubeLive"

  Did those API calls continue to work, unmodified, with the "version=1" parameter still in them?

Thanks again!

[tfabris]

Also, @quog - attached to this message is a test script containing a prototype of the "fixed" version of the WebApiAuth function. Would you mind helping me out, and running this test file from the console and sending me its output? It should run from a Linux or Mac shell prompt, and it should be run from a computer on the same network as the NAS. You have to edit three variables at the top of the file to point to your server running DSM 7.x, and its username, rename the file from .txt to .sh, and chmod it to be executable. The output should look similar this, except yours should try to authenticate to version 7 instead of version 6:

DSM Authentication Test - Querying http://192.168.0.88:5000/webapi to see if it is DSM 6.x.
DSM Authentication Test - Version Response: {"data":{"SYNO.API.Auth":{"maxVersion":6,"minVersion":1,"path":"auth.cgi"}},"success":true}.
DSM Authentication Test - DSM version appears to be 6 - Result: {"data":{"SYNO.API.Auth":{"maxVersion":6,"minVersion":1,"path":"auth.cgi"}},"success":true}.
DSM Authentication Test - Querying http://192.168.0.88:5000/webapi to see if it is DSM 7.x.
DSM Authentication Test - Query did not succeed - Result: {"error":{"code":102},"success":false}.
DSM Authentication Test - Attempting to authenticate to DSM version 6.
DSM Authentication Test - Auth response: {"success":true}.

Script should be attached below. Thanks!
DsmAuthTest.txt

[tfabris]

Also, @quog - Can you tell if logging to the Synology log (the function "LogMessage" in CrowCamHelperFunctions.sh) is working on DSM 7.x?

[tfabris]

OK, I believe I've made the necessary changes to the master branch code to make it work under DSM 7.x, as long as it's run as "root".

I was able to do some limited testing on DSM 7.2 by logging into their online "demo" of DSM 7.2 from the Synology main web site. I see that the logging function still works, my tests to detect DSM6 or DSM7 are working, etc. I wasn't able to actually test whether the authentication API command works, though, because when you log into their online demo, you don't know the password for the user "demouser" and it terminates your session if you try to add new users or change the demouser password.

I haven't yet updated the "ping" command, because I have strong feelings about it needing to truly be a "ping" and not some other command. The trick of using nslookup instead of ping is a good trick, but it's not exactly the same thing. In particular, I'd like to have the option to change the configuration so that it pings some other server other than a DNS server to perform the network test. There could theoretically be situations where you would want that. Leaving it as a true ping gives it that flexibility.

@quog - If you are able, please try the new master branch version of the CrowCam code on your 7.2 server and tell me if it's working OK for you. The only important changed files are "CrowCam.sh" and "CrowCamHelperFunctions.sh", and you can leave your configuration files as they are. But you will need to reconfigure the tasks in Task Scheduler so that they run under the user "root" in order for the ping command to work.

[quog]

Hi Tony,

If the scripts had been configured to run from the Synology Task Scheduler as "root" with the "bash" command line (as described in the configuration instructions here), then would the existing "ping" command have worked without modification?

Yes - note not tested, but I could ping as root from an SSH connection to the NAS

After you modified the "WebApiAuth" function to use "entry.cgi" and "version=3" to log in, did the other API commands still work without modification?

Yes - I made no other modifications. Regards Damian Damian Douglas-Meyer 0414 419 624 ***@***.*** Damian Douglas-Meyer 0414 419 624 ***@***.***

…

On Wed, 17 Jul 2024 at 04:33, Tony Fabris ***@***.***> wrote: @quog - Understood, thanks so much! Your comments are very helpful. Since my NAS is still running DSM 6.x, I don't have a way to test on DSM 7.x at the moment. My intention is to upgrade my NAS to a newer model which is already running DSM 7.x, and then migrate the data and scripts over to it, but I haven't done that yet. I've updated the code to remove synoservicectl already. Regarding the other two changes... Clarifying questions: If the scripts had been configured to run from the Synology Task Scheduler as "root" with the "bash" command line (as described in the configuration instructions here), then would the existing "ping" command have worked without modification? After you modified the the "WebApiAuth" function to use "entry.cgi" and "version=3" to log in, did the other API commands still work without modification? Examples: CrowCamHelperFunctions.sh "IsStreamRunning" "StartStream" "StopStream" "CreateNewStream" CrowCam.sh Three instances of calls to "entry.cgi?api=SYNO.SurveillanceStation.YoutubeLive" Did those API calls continue to work, unmodified, with the "version=1" parameter still in them? Thanks again! — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[quog]

Hi Tony, The tests for DSM version in that script worked, but the auth didn't - which I can't figure out as the same code works in CrowCam.sh - perhaps some other instantiation is needed? My settings where the same as in the live CrowCam.sh (with the correct password of course.) username="apiaccess" password="REDACTED" webApiRootUrl="http://127.0.0.1:5000/webapi" DSM Authentication Test - Querying http://127.0.0.1:5000/webapi to see if it is DSM 6.x. DSM Authentication Test - Version Response: {"data":{"SYNO.API.Auth":{"maxVersion":7,"minVersion":1,"path":"entry.cgi"}},"success":true}. DSM Authentication Test - DSM version appears to be 6 - Result: {"data":{"SYNO.API.Auth":{"maxVersion":7,"minVersion":1,"path":"entry.cgi"}},"success":true}. DSM Authentication Test - Querying http://127.0.0.1:5000/webapi to see if it is DSM 7.x. DSM Authentication Test - DSM version appears to be 7 - Result: {"data":{"SYNO.API.Auth":{"maxVersion":7,"minVersion":1,"path":"entry.cgi"}},"success":true}. DSM Authentication Test - Attempting to authenticate to DSM version 7. http://127.0.0.1:5000/webapi/entry.cgi?api=SYNO.API.Auth&version=3&method=login&account=apiaccess&passwd=REDACTED&session=SurveillanceStation&format=cookie DSM Authentication Test - Auth response: {"error":{"code":400},"success":false}. DSM Authentication Test - The call to authenticate with the Synology Web API failed. Exiting program. Response: {"error":{"code":400},"success":false}. Terminated Weird - I will look at it again later. Regards Damian Damian Douglas-Meyer 0414 419 624 ***@***.***

…

On Wed, 17 Jul 2024 at 07:20, Tony Fabris ***@***.***> wrote: Also, @quog <https://github.com/quog> - attached to this message is a test script containing a prototype of the "fixed" version of the WebApiAuth function. Would you mind helping me out, and running this test file from the console and sending me its output? It should run from a Linux or Mac shell prompt, and it should be run from a computer on the same network as the NAS. You have to edit three variables at the top of the file to point to your server running DSM 7.x, and its username, rename the file from .txt to .sh, and chmod it to be executable. The output should look similar this, except yours should try to authenticate to version 7 instead of version 6: DSM Authentication Test - Querying http://192.168.0.88:5000/webapi to see if it is DSM 6.x. DSM Authentication Test - Version Response: {"data":{"SYNO.API.Auth":{"maxVersion":6,"minVersion":1,"path":"auth.cgi"}},"success":true}. DSM Authentication Test - DSM version appears to be 6 - Result: {"data":{"SYNO.API.Auth":{"maxVersion":6,"minVersion":1,"path":"auth.cgi"}},"success":true}. DSM Authentication Test - Querying http://192.168.0.88:5000/webapi to see if it is DSM 7.x. DSM Authentication Test - Query did not succeed - Result: {"error":{"code":102},"success":false}. DSM Authentication Test - Attempting to authenticate to DSM version 6. DSM Authentication Test - Auth response: {"success":true}. Script should be attached below. Thanks! DsmAuthTest.txt <https://github.com/user-attachments/files/16257475/DsmAuthTest.txt> — Reply to this email directly, view it on GitHub <#77 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB5HTO3J3254F6XFEYZTUSDZMWTDLAVCNFSM6AAAAABK326XRKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZRHE3TGMJUGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[quog]

Yes - worked fine. Damian Douglas-Meyer 0414 419 624 ***@***.***

…

On Thu, 18 Jul 2024 at 01:06, Tony Fabris ***@***.***> wrote: Also, @quog - Can you tell if logging to the Synology log (the function "LogMessage" in CrowCamHelperFunctions.sh) is working on DSM 7.x? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>