SQLAlchemy error on old cached database

[AyeshaKhalidTST]

Bug description

ragna init

Select all storages and both ChatGPT assistants, then

OPENAI_API_KEY="..." ragna ui

Try to add a PDF document and start a chat. I've tried many different documents, but they result in the same error.

ragna.log

How to reproduce the bug?

Try to add a PDF document and start a chat.

Versions and dependencies used

conda create -n chatbot python=3 && pip install 'ragna[all]'

# packages in environment at /Users/akhalid/mambaforge/envs/chatbot:
#
# Name                    Version                   Build  Channel
aiofiles                  23.2.1                   pypi_0    pypi
annotated-types           0.6.0                    pypi_0    pypi
anyio                     4.3.0                    pypi_0    pypi
asgiref                   3.8.1                    pypi_0    pypi
attrs                     23.2.0                   pypi_0    pypi
backoff                   2.2.1                    pypi_0    pypi
bcrypt                    4.1.2                    pypi_0    pypi
bleach                    6.1.0                    pypi_0    pypi
bokeh                     3.3.4                    pypi_0    pypi
build                     1.2.1                    pypi_0    pypi
bzip2                     1.0.8                h93a5062_5    conda-forge
ca-certificates           2024.2.2             hf0a4a13_0    conda-forge
cachetools                5.3.3                    pypi_0    pypi
certifi                   2024.2.2                 pypi_0    pypi
charset-normalizer        3.3.2                    pypi_0    pypi
chroma-hnswlib            0.7.3                    pypi_0    pypi
chromadb                  0.4.24                   pypi_0    pypi
click                     8.1.7                    pypi_0    pypi
coloredlogs               15.0.1                   pypi_0    pypi
contourpy                 1.2.1                    pypi_0    pypi
decorator                 5.1.1                    pypi_0    pypi
deprecated                1.2.14                   pypi_0    pypi
deprecation               2.1.0                    pypi_0    pypi
emoji                     2.11.0                   pypi_0    pypi
fastapi                   0.110.1                  pypi_0    pypi
filelock                  3.13.4                   pypi_0    pypi
flatbuffers               24.3.25                  pypi_0    pypi
fsspec                    2024.3.1                 pypi_0    pypi
google-auth               2.29.0                   pypi_0    pypi
googleapis-common-protos  1.63.0                   pypi_0    pypi
grpcio                    1.62.1                   pypi_0    pypi
h11                       0.14.0                   pypi_0    pypi
httpcore                  1.0.5                    pypi_0    pypi
httptools                 0.6.1                    pypi_0    pypi
httpx                     0.27.0                   pypi_0    pypi
httpx-sse                 0.4.0                    pypi_0    pypi
huggingface-hub           0.22.2                   pypi_0    pypi
humanfriendly             10.0                     pypi_0    pypi
idna                      3.7                      pypi_0    pypi
ijson                     3.2.3                    pypi_0    pypi
importlib-metadata        7.0.0                    pypi_0    pypi
importlib-resources       6.4.0                    pypi_0    pypi
jinja2                    3.1.3                    pypi_0    pypi
kubernetes                29.0.0                   pypi_0    pypi
lancedb                   0.6.8                    pypi_0    pypi
libexpat                  2.6.2                hebf3989_0    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libsqlite                 3.45.2               h091b4b1_0    conda-forge
libzlib                   1.2.13               h53f4e23_5    conda-forge
linkify-it-py             2.0.3                    pypi_0    pypi
lxml                      5.2.1                    pypi_0    pypi
markdown                  3.6                      pypi_0    pypi
markdown-it-py            3.0.0                    pypi_0    pypi
markupsafe                2.1.5                    pypi_0    pypi
mdit-py-plugins           0.4.0                    pypi_0    pypi
mdurl                     0.1.2                    pypi_0    pypi
mmh3                      4.1.0                    pypi_0    pypi
monotonic                 1.6                      pypi_0    pypi
mpmath                    1.3.0                    pypi_0    pypi
ncurses                   6.4.20240210         h078ce10_0    conda-forge
numpy                     1.26.4                   pypi_0    pypi
oauthlib                  3.2.2                    pypi_0    pypi
onnxruntime               1.17.1                   pypi_0    pypi
openssl                   3.2.1                h0d3ecfb_1    conda-forge
opentelemetry-api         1.24.0                   pypi_0    pypi
opentelemetry-exporter-otlp-proto-common 1.24.0                   pypi_0    pypi
opentelemetry-exporter-otlp-proto-grpc 1.24.0                   pypi_0    pypi
opentelemetry-instrumentation 0.45b0                   pypi_0    pypi
opentelemetry-instrumentation-asgi 0.45b0                   pypi_0    pypi
opentelemetry-instrumentation-fastapi 0.45b0                   pypi_0    pypi
opentelemetry-proto       1.24.0                   pypi_0    pypi
opentelemetry-sdk         1.24.0                   pypi_0    pypi
opentelemetry-semantic-conventions 0.45b0                   pypi_0    pypi
opentelemetry-util-http   0.45b0                   pypi_0    pypi
orjson                    3.10.0                   pypi_0    pypi
overrides                 7.7.0                    pypi_0    pypi
packaging                 24.0                     pypi_0    pypi
pandas                    2.2.1                    pypi_0    pypi
panel                     1.3.8                    pypi_0    pypi
param                     2.1.0                    pypi_0    pypi
pillow                    10.3.0                   pypi_0    pypi
pip                       24.0               pyhd8ed1ab_0    conda-forge
posthog                   3.5.0                    pypi_0    pypi
prompt-toolkit            3.0.36                   pypi_0    pypi
protobuf                  4.25.3                   pypi_0    pypi
pulsar-client             3.4.0                    pypi_0    pypi
py                        1.11.0                   pypi_0    pypi
pyarrow                   15.0.0                   pypi_0    pypi
pyasn1                    0.6.0                    pypi_0    pypi
pyasn1-modules            0.4.0                    pypi_0    pypi
pydantic                  2.6.4                    pypi_0    pypi
pydantic-core             2.16.3                   pypi_0    pypi
pydantic-settings         2.2.1                    pypi_0    pypi
pygments                  2.17.2                   pypi_0    pypi
pyjwt                     2.8.0                    pypi_0    pypi
pylance                   0.10.10                  pypi_0    pypi
pymupdf                   1.24.1                   pypi_0    pypi
pymupdfb                  1.24.1                   pypi_0    pypi
pypika                    0.48.9                   pypi_0    pypi
pyproject-hooks           1.0.0                    pypi_0    pypi
python                    3.12.2          hdf0ec26_0_cpython    conda-forge
python-dateutil           2.9.0.post0              pypi_0    pypi
python-docx               1.1.0                    pypi_0    pypi
python-dotenv             1.0.1                    pypi_0    pypi
python-multipart          0.0.9                    pypi_0    pypi
python-pptx               0.6.23                   pypi_0    pypi
pytz                      2024.1                   pypi_0    pypi
pyviz-comms               3.0.2                    pypi_0    pypi
pyyaml                    6.0.1                    pypi_0    pypi
questionary               2.0.1                    pypi_0    pypi
ragna                     0.2.0                    pypi_0    pypi
ratelimiter               1.2.0.post0              pypi_0    pypi
readline                  8.2                  h92ec313_1    conda-forge
redis                     5.0.3                    pypi_0    pypi
regex                     2023.12.25               pypi_0    pypi
requests                  2.31.0                   pypi_0    pypi
requests-oauthlib         2.0.0                    pypi_0    pypi
retry                     0.9.2                    pypi_0    pypi
rich                      13.7.1                   pypi_0    pypi
rsa                       4.9                      pypi_0    pypi
semver                    3.0.2                    pypi_0    pypi
setuptools                69.2.0             pyhd8ed1ab_0    conda-forge
shellingham               1.5.4                    pypi_0    pypi
six                       1.16.0                   pypi_0    pypi
sniffio                   1.3.1                    pypi_0    pypi
sqlalchemy                2.0.29                   pypi_0    pypi
starlette                 0.37.2                   pypi_0    pypi
sympy                     1.12                     pypi_0    pypi
tenacity                  8.2.3                    pypi_0    pypi
tiktoken                  0.6.0                    pypi_0    pypi
tk                        8.6.13               h5083fa2_1    conda-forge
tokenizers                0.15.2                   pypi_0    pypi
tomlkit                   0.12.4                   pypi_0    pypi
tornado                   6.4                      pypi_0    pypi
tqdm                      4.66.2                   pypi_0    pypi
typer                     0.12.3                   pypi_0    pypi
typing-extensions         4.11.0                   pypi_0    pypi
tzdata                    2024.1                   pypi_0    pypi
uc-micro-py               1.0.3                    pypi_0    pypi
urllib3                   2.2.1                    pypi_0    pypi
uvicorn                   0.29.0                   pypi_0    pypi
uvloop                    0.19.0                   pypi_0    pypi
watchfiles                0.21.0                   pypi_0    pypi
wcwidth                   0.2.13                   pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
websocket-client          1.7.0                    pypi_0    pypi
websockets                12.0                     pypi_0    pypi
wheel                     0.43.0             pyhd8ed1ab_1    conda-forge
wrapt                     1.16.0                   pypi_0    pypi
xlsxwriter                3.2.0                    pypi_0    pypi
xyzservices               2024.4.0                 pypi_0    pypi
xz                        5.2.6                h57fd34a_0    conda-forge
zipp                      3.18.1                   pypi_0    pypi

Anything else?

No response

[pmeier]

Traceback (most recent call last):
  File "/Users/akhalid/mambaforge/envs/chatbot/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1971, in _exec_single_context
    self.dialect.do_execute(
  File "/Users/akhalid/mambaforge/envs/chatbot/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 919, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: no such column: sources.content

This looks like the aftermath of #264 that was released with 0.2.0. Was the database you are using created by 0.1.x? The default location should be ~/.cache/ragna/ragna.db. Deleting it should resolve your issue.

[AyeshaKhalidTST]

Thanks, that solved the issue. Should perhaps a cache invalidation be part of the first-time run when the schema changes?

[pmeier]

Yeah, we should probably catch this error and throw a more descriptive one. Actually deleting anything is something that we can't do, because you can hook any database into Ragna, not just sqllite local ones. But even if we could, we shouldn't do destructive operations by default.