General speed enhancements required:
- Cut down ram usage per batch
- Switch pandas df data to numpy and vectorise operations
- Move binana & ecifs to a single distance matrix calculation
- Remove redundant feature processing
- Remove redundant scaler and feature list loading
General speed enhancements required: