Add selectable dictionary sources and fix unreliable word lookups - #6
Open
hassanraza464 wants to merge 3 commits into
Open
Add selectable dictionary sources and fix unreliable word lookups#6hassanraza464 wants to merge 3 commits into
hassanraza464 wants to merge 3 commits into
Conversation
The Free Dictionary API intermittently returns 5xx errors, and returns its ordinary "No Definitions Found" 404 for words it has no entry for. That 404 is indistinguishable from a word not existing, so common words like "is" and "be" were reported as not being words at all. Reliability: * Retry 500/502/503/504 and connection errors up to three times with a short backoff. 404 still fails immediately, so unknown words stay responsive. * Speak a message for an exhausted 5xx rather than the raw urllib error. * Verify a 404 against Wiktionary before reporting a word as unknown, so "unable to find definition" is only spoken when both sources agree. Requires an "en" entry, so pages that exist for other languages are not mistaken for English words. Word selection: * Strip punctuation from the ends of a word before looking it up, so words adjacent to punctuation resolve. Internal apostrophes and hyphens are kept, and the unstripped form is tried as a fallback for words like 'tis. Text that is only punctuation no longer triggers a lookup. Type detection still runs on the original text, leaving IP, URL and ISBN handling unchanged. Sources: * Add a provider abstraction returning normalised entries, so the formatting and audio code is independent of where a definition came from. * Add Wiktionary as a selectable source, including HTML stripping, filtering to English entries and flattening its nested senses. Examples are allocated across each part of speech so a heading does not take the sentence belonging to the sense beneath it. * Add a settings panel to choose the dictionary, plus an optional fallback to the other source when the chosen one lacks a definition or pronunciation. Defaults leave existing behaviour unchanged. * Fall back to Wikimedia Commons pronunciations, using Wikimedia's mp3 transcodes so the existing player handles them, when a recording cannot be played. Report a message instead of only beeping when none can be. * Request Wikimedia responses gzipped, which reduces a large entry from 69KB to around 9KB. Also sync manifest.ini with the version already set in buildVars.py.
Covers the dictionary selection, the fallback option, punctuation handling and
the "definition unavailable" message in the readme, and fills in the five new
strings for the Spanish, French and Russian catalogues. The settings panel title
reuses the existing "Text Information" message, so it needed no new translation.
Two entries, "Definition for {0}" and "Play {0}", were already marked fuzzy
before this change and are left as they are.
The dictionary API has been seen returning an empty body with a 200 status while it is unwell. Parsing happened outside the try, so the lookup thread died with a JSONDecodeError and the user was told nothing at all. Parsing now happens inside it, and an unusable body is reported like any other failure. This also covers a response that is valid json but an empty list, which raised IndexError. Looking elsewhere for a pronunciation ran between the beep and the definition being spoken, so a lookup paused for about half a second before saying anything. It happens whenever the fallback option is enabled and the dictionary carries no recording for a word, which is why it was intermittent. The definition is now spoken first and the recording fetched afterwards. That leaves a short window where a definition has been spoken but its audio has not arrived, so opening the dialog within it shows no play buttons. Reaching that needs a double press faster than the lookup, and the buttons appear on a second attempt.
Author
|
Pushed two fixes found while testing:
The second leaves a short window where a definition has been spoken but its audio has not arrived, so opening the dialog inside it shows no play buttons. Reaching it needs a double press faster than the lookup, and a second attempt has them. Happy to close that properly if you would rather, though every option I looked at trades it for either a delayed dialog or one that grows buttons while being read. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi, thanks for the add-on. While using it I kept hitting
HTTP Error 502: Bad Gatewayon word lookups, and the digging turned up a second, quieter problem. This fixes both, then adds a way to avoid depending on a single dictionary.Happy to split this into smaller PRs, drop the feature half, or adjust anything, just say.
The 502s
api.dictionaryapi.devreturns 200/404/502 nondeterministically per request. Their Cloudflare layer caches withmax-age=14400, so a failure is served as a cacheHITfor hours and looks deterministic; forcing aMISSwith a cache buster showed the origin itself is unstable. Reproducible from unrelated networks, so it is not local to me.The quieter problem
is,wasandbereturn 404 from a fresh origin request, with the API's own{"title":"No Definitions Found",...}body, byte identical to what a nonsense word returns. The add-on cannot distinguish "no entry" from "not a word", so it was speaking "unable to find definition for word" foris. That behaviour survives the outage, since it is a property of the data rather than the downtime.Changes
Reliability:
enkey, so pages existing only for other languages (e.g.hunde) are not mistaken for English words.Word selection:
let's,so-so), the unstripped form is tried as a fallback for'tisande.g., and text that is only punctuation no longer triggers a lookup. Type detection still runs on the original text, so the IP, URL and ISBN paths are untouched.Sources:
format_word_meaningand the audio dialog are unchanged.<ol>, which duplicates every child inside its parent, and a parent'sparsedExamplesaggregates all of its children's (13 forbeitem 1, 31 for item 8), so examples are allocated per part of speech rather than taking the first available.Also syncs
manifest.iniwith the version already set inbuildVars.py, adds the two new modules topythonSourcesso their strings are extracted, documents the settings in the readme, and fills in the new strings for es/fr/ru. I am not a native speaker of those, so please treat the translations as a starting point, or drop them if you would rather they went through the translators. No version bump, as that is your call.Testing
Manually against NVDA on Windows 11, with both sources, the fallback on and off, and during the live outage. Verified the previously broken words (
is,be,was) now return definitions from Wiktionary, that punctuation-adjacent selections resolve, thatlet'sandso-soare unaffected, and that Commons audio substitutes when the dictionary's mp3 fails. Two pre-existing fuzzy entries, "Definition for {0}" and "Play {0}", are left as they were.