Program still runs slow

[JarrahGos]

Check ram usage on the RPI. If a signifficant amount is free, consider loading the whole database into ram and working that way. (load it into a hast table?)

If little ram, overclocking or new hardware will be the only option.

[Tinkerlad0]

Has using a sql database instead of a flat file been considered? could help with this if it can manage to cache more effectively.

[JarrahGos]

I did initially consider it. Not sure whether it would work on the pi though.
There isn't much RAM to play with and the overhead of an DBMS could make things worse.

In saying that, it may also make things far better. We'd have to test it.