No scrobble after Trakt 12-11-2022 DB Crash due to corrupted watched_backlog.json file

[yeyogrande]

To RECAP: for any users who are experiencing NO scrobble issues after Trakt December 21, 2022 crash. I had a corrupted watched_backlog.json file. It was creating many multiple episode instances of episodes in my trakt history. And, it was also causing the scrobbler program to crash. I moved the watched_backlog.json file and did a trakts start --restart and it was fixed. For recovery of missed trakt history read the rest of this post.

No scrobbles since Dec 11 2022 Trakt DB crash. No smplayer, vlc, mpv scrobble anymore. Tried commands auth init etc. Tried re-installing current newest release & re configed it. No scrobble in players. Tried reinstalling newest beta client as of Jan 2023. Tried logging out back in trakt. Tried all new token and auth commands all to no avail. No scrobbling in smplayer, mpv or vlc. I cant be the only one experiencing this? Am I? the only one?. Thank You and LOVE this app, it was truly heaven sent!! Never a problem until THE BIG CRASH of XMAS 2022 at Trakt. Please advise.

Lubuntu 20.04
Newest versions of everything updated through commandline sudo apt update & upgrade as of Jan 10

[yeyogrande]

I had just run these commands again:

trakts auth --force to get a new API key
trakts start --restart to restart the app
(Optional) trakts backlog clear

got this error from backlog command:
trakts backlog clear

KeyError

'season'

at .local/pipx/venvs/trakt-scrobbler/lib/python3.8/site-packages/trakt_scrobbler/trakt_interface.py:139 in prepare_history_data
135│ return {'movies': [{'ids': ids, 'watched_at': watched_at}]}
136│ else: # TODO: Group data by show instead of sending episode-wise
137│ return {'shows': [
138│ {'ids': ids, 'seasons': [
→ 139│ {'number': media_info['season'], 'episodes': [
140│ {'number': media_info['episode'], 'watched_at': watched_at}]
141│ }]
142│ }]
143│ }

status command gives:
trakts status
The scrobbler is running
Monitored players: mpv, plex, smplayer@mpv, vlc
No activity yet.
Which also verifies my plex only scrobbling coming from different scrobbler project.

grepped /var/log/syslog authlog dmesg and boot logs for 401 scrobbler error to no avail.
Did several login logouts to trakt site as well as several trakts restarts. See one guy who said he got sucess after 2 forced re auth commands. will try again a third time. I had not seen that closed ticket with the guy with the 401 scrobbler error after the crash til today when I checked back to see if anyone responded to this ticket a few min ago. So I do see I was not the only one. Though very very mild consolation was felt ;)
Once again your scrobbler was aces until this my first hiccup with it. Been running for a decent while. Do you suggest I move back to last proper release as I did reininstall with beta: Trakts version 1.6.1-beta1

[iamkroot]

@boris377 have you tried the instructions in #234?

[iamkroot]

@yeyogrande Without proper logs showing scrobble errors, I can't diagnose the issue. Please upload a sufficiently redacted version of the log file (trakts log open). Running the force auth as specified in #234 was enough to get the scrobbler working again in my case (running Manjaro linux) - you don't need to switch to older releases.

Regarding the error you see in backlog clear output, I suspect some file has been edited manually. The "season" number is missing from one of the scrobbled episodes present in your backlog, which is not supposed to happen unless the backlog is edited by hand. Would be useful if you could upload the trakt_backlog.json (redacting private data, preserving the structure) that lives alongside the log files.

[yeyogrande]

Thank you for timely reply iamkroot ...you the man :) [that i wish i was ;) ]
Starting troubleshooting again right now following bugtraq #234:
Closed all trakt web page instances. I Go to https://trakt.tv/oauth/authorized_applications
X REVOKE ACCESS (4th Try since trakt DB crash !)
trakts -V
Trakts version 1.6.1-beta1
:~$ trakts auth --force
Forcing trakt authentication
Open https://trakt.tv/activate in your browser and enter this code: xxxxxx
Opening in existing browser session.
App authorized successfully.
Token valid until 04/06/23
trakts start --restart
The scrobbler is running
Monitored players: mpv, plex, smplayer@mpv, vlc
No activity yet.

Tried to scrobble a show. No scrobble. Logout and login Trakt website. Try again. No scrobble.
trakts status
The scrobbler is running
Monitored players: mpv, plex, smplayer@mpv, vlc
No activity yet

trakts log open:
Log file is located at: "/myhomedir/.local/share/trakt-scrobbler/trakt_scrobbler.log"
In case this command doesn't work, manually open the log file from the path.
mybox:~$ Warning: program returned non-zero exit code #256
Opening "/myhome/.local/share/trakt-scrobbler/trakt_scrobbler.log" with FeatherPad (text/x-log)
Icon theme "elementary" not found.

Here is the only piece of log that has Unhandled exception Traceback and line number references, that overall looks good:

2023-01-04 16:36:31,757 - ERROR - backlog_cleaner - init - Unhandled exception
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 1254, in run
self.function(*self.args, **self.kwargs)
File "/home/stax/.local/pipx/venvs/trakt-scrobbler/lib/python3.8/site-packages/trakt_scrobbler/backlog_cleaner.py", line 50, in clear
if trakt.add_to_history(**item):
File "/home/stax/.local/pipx/venvs/trakt-scrobbler/lib/python3.8/site-packages/trakt_scrobbler/trakt_interface.py", line 148, in add_to_history
history = prepare_history_data(watched_at, media_info)
File "/home/stax/.local/pipx/venvs/trakt-scrobbler/lib/python3.8/site-packages/trakt_scrobbler/trakt_interface.py", line 139, in prepare_history_data
{'number': media_info['season'], 'episodes': [
KeyError: 'season'
2023-01-06 12:12:41,020 - DEBUG - MainThread - notifier - Notifications enabled for categories: exception, misc, scrobble.pause, scrobble.resume, scrobble.start, scrobble.stop, trakt
2023-01-06 12:12:41,229 - INFO - notify_loop - notifier - Starting notif loop
2023-01-06 12:12:41,229 - DEBUG - MainThread - notifier - Notif actions enabled for categories: exception, misc, scrobble.pause, scrobble.resume, scrobble.start, scrobble.stop, trakt
2023-01-06 12:12:41,231 - INFO - MainThread - trakt_auth - Access token not found. Initiating device authentication.
2023-01-06 12:12:41,563 - INFO - MainThread - trakt_auth - Verification URL: https://trakt.tv/activate
2023-01-06 12:12:41,564 - INFO - MainThread - trakt_auth - User Code: 9344835A
2023-01-06 12:12:52,901 - WARNING - MainThread - utils - Request failed
2023-01-06 12:12:52,902 - DEBUG - MainThread - utils - Request: post {'url': 'https://api.trakt.tv/oauth/device/token', 'headers': {'Content-Type': 'application/json'}, 'json': {'code': '145a6deea008f3c08f3336b6a7e9689d1f8a16433864b8015e9f9d22123a1e44', 'client_id': 'ab0133a2365b2c64d70fd2adf3c7e775a4131471b56340933335af1b94785a3a', 'client_secret': 'b574acd5857310fcdc1e195c5953795fc61a1d89d69fec1649624d54cb666222'}}
2023-01-06 12:12:52,902 - DEBUG - MainThread - utils - Response: <Response [400]>
2023-01-06 12:12:52,902 - DEBUG - MainThread - trakt_auth - Waiting for user to authorize the app.
2023-01-06 12:12:57,967 - WARNING - MainThread - utils - Request failed
2023-01-06 12:12:57,967 - DEBUG - MainThread - utils - Request: post {'url': 'https://api.trakt.tv/oauth/device/token', 'headers': {'Content-Type': 'application/json'}, 'json': {'code': '145a6deea008f3c08f3336b6a7e9689d1f8a16433864b8015e9f9d22123a1e44', 'client_id': 'ab0133a2365b2c64d70fd2adf3c7e775a4131471b56340933335af1b94785a3a', 'client_secret': 'b574acd5857310fcdc1e195c5953795fc61a1d89d69fec1649624d54cb666222'}}
2023-01-06 12:12:57,967 - DEBUG - MainThread - utils - Response: <Response [400]>
2023-01-06 12:12:57,967 - DEBUG - MainThread - trakt_auth - Waiting for user to authorize the app.
2023-01-06 12:13:03,045 - WARNING - MainThread - utils - Request failed
2023-01-06 12:13:03,045 - DEBUG - MainThread - utils - Request: post {'url': 'https://api.trakt.tv/oauth/device/token', 'headers': {'Content-Type': 'application/json'}, 'json': {'code': '145a6deea008f3c08f3336b6a7e9689d1f8a16433864b8015e9f9d22123a1e44', 'client_id': 'ab0133a2365b2c64d70fd2adf3c7e775a4131471b56340933335af1b94785a3a', 'client_secret': 'b574acd5857310fcdc1e195c5953795fc61a1d89d69fec1649624d54cb666222'}}
2023-01-06 12:13:03,046 - DEBUG - MainThread - utils - Response: <Response [400]>
2023-01-06 12:13:03,046 - DEBUG - MainThread - trakt_auth - Waiting for user to authorize the app.
2023-01-06 12:13:08,137 - INFO - MainThread - trakt_auth - App authorized successfully.
2023-01-06 12:15:57,408 - DEBUG - MainThread - notifier - Notifications enabled for categories: exception, misc, scrobble.pause, scrobble.resume, scrobble.start, scrobble.stop, trakt
2023-01-06 12:15:57,437 - INFO - notify_loop - notifier - Starting notif loop
2023-01-06 12:15:57,438 - DEBUG - MainThread - notifier - Notif actions enabled for categories: exception, misc, scrobble.pause, scrobble.resume, scrobble.start, scrobble.stop, trakt
2023-01-06 12:16:00,742 - DEBUG - MainThread - backlog_cleaner - Adding item to history {'state': 0, 'progress': 99.96, 'media_info': {'type': 'episode', 'title': 'The Young Pope', 'season': 1, 'episode': 5}, 'updated_at': 1670816993.0620918}
2023-01-06 12:16:01,026 - INFO - MainThread - backlog_cleaner - Successfully added media to history.
2023-01-06 12:16:01,026 - DEBUG - MainThread - backlog_cleaner - Adding item to history {'state': 0, 'progress': 99.96, 'media_info': {'type': 'episode', 'title': 'The Young Pope', 'season': 1, 'episode': 5}, 'updated_at': 1670816993.0646179}
2023-01-06 12:16:01,178 - INFO - MainThread - backlog_cleaner - Successfully added media to history.
2023-01-06 12:16:01,179 - DEBUG - MainThread - backlog_cleaner - Adding item to history {'state': 0, 'progress': 100.0, 'media_info': {'type': 'episode', 'title': 'The Young Pope', 'season': 1, 'episode': 1}, 'updated_at': 1670822083.8170059}

MOST OF LOG looks like above last lines from line 99.96 on down. I take it this is proper, correct?
Do I need to look further back in older logs?

you said in your response to upload trakt_backlog.json: I don't have that file verbatim
mybox/.local/share/trakt-scrobbler$ ls
plex_token.txt trakt_scrobbler.log trakt_scrobbler.log.2 trakt_scrobbler.log.4 trakt_token.json
trakt_cache.json trakt_scrobbler.log.1 trakt_scrobbler.log.3 trakt_scrobbler.log.5 watched_backlog.json
So I took a look at :
mybox:~/.local/share/trakt-scrobbler$ nano watched_backlog.json (Is this the file you are refering to?)
This file: watched_backlog.json looks pretty good do you want me to grep for something?

Also, regarding what you said above:
iamkroot said: Regarding the error you see in backlog clear output, I suspect some file has been edited manually. The "season" number is missing from one of the scrobbled episodes present in your backlog, which is not supposed to happen unless the backlog is edited by hand.

I respond: I have not edited these files nor had I looked at them as I was ignorant of them since I had been running your scrobbler successfully up through many upgraded versions. I.E. no problems, no looky at the loggies. Thanks.

Thanks again brother. Lemme know if I can upload anything else. Peace out.

[yeyogrande]

Tried a fresh Trakt account. Still no scrobble. :(
BTW I was at version 1.4.2 when DB crashed on 12-11-2022
Then tried 1.6.0 and now 1.6.1-beta1

[iamkroot]

This file: watched_backlog.json looks pretty good do you want me to grep for something?

Would be helpful if you could paste a redacted version of it here (removing titles).

Next steps:

1. Read what watched_backlog.json is used for - here. In short, it stored all the failed scrobbles when the Trakt API was down. Now we try to sync it back to Trakt, but one (or more) of those scrobbles is corrupted for some reason and our program crashes midway. Next time we start, we again try to sync and fail again. This is why you are seeing duplicate entries on your Trakt history (from the scrobbles that did get synced).
2. Manually mark the scrobbles from watched_backlog.json on Trakt using their "Add to History" button on the website.
   • For the duplicated scrobbles, remove them from Trakt
   • To get the watch time, use the updated_at value for each scrobble. On Linux, you can convert it to human readable form using date -d. Example: date -d @1672248561.234
   • Once you are satisfied that your entire history is present on Trakt correctly, delete the watched_backlog.json file.
3. After restarting with trakts start -r, your scrobbler should work normally.

(No need to create a second Trakt account)

[yeyogrande]

Sounds promising :)

Well, how about I just mv the backlog.json file and restart trakts? Can I deal with history later by switching the newly created backlog.json out for my mv'ed file and then reverse the process? I would like to see a notification that my scrobbling is working? Thanks for all iamkroot. YG.

couple minutes later....

I couldn't wait I had to mv the file and I saw that lovely scrobble notification !!! :)
You truly are THE MAN!!!!

So, I guess you really want me to just look through the file and edit trakt by hand huh? Sounds laborious, but if this is the only recourse, so be it. Maybe comment on above b4 you close this ticket. Thanks a million. YG.

P.P.S. I posted this recap in the beginning of first post in this issue:

To RECAP: for any users who are experiencing NO scrobble issues after Trakt December 21, 2022 crash. I had a corrupted watched_backlog.json file. It was creating many multiple episode instances of episodes in my trakt history. And, it was also causing the scrobbler program to crash. I moved the watched_backlog.json file and did a trakts start --restart and it was fixed. For recovery of missed trakt history read the rest of this post.

[yeyogrande]

think i answered some of my own questions above. did a wc -l is 2749 lines. on backlog finding date with date -d (1575 is Dec22 1st episode and it starts at 1566) and doing a piece at a time. will report success or failure back soon...
I got 5 more days restored with lots of multiples, thought it would be easier, should have written it down step by step tried to keep it all in my head, got confused after a while. Had enough tonite :) See why you did not suggest this method.

[yeyogrande]

Thanks again for keeping me from having to RTFM.

It did occur to me that if the backlog was saved off into smaller chunks (say by days for instance) then they could be restored with a backlog restore all option. Or also in chunks by days with a different option. To make catastrophic restores easier. Of course if the user knows what the hell he is doing he will never get such a large backlog as I did. Humbly Yours, YG.