Skip to content

AzureDirectory.DeleteFile() never evicts the local cache copy #39

Description

@lethallynx

When a file-system cache directory is supplied (e.g. new AzureDirectory(client, catalog, new SimpleFSDirectory(tempDir), true)), AzureDirectory.DeleteFile(name) deletes the blob and the in-memory metadata entry but never deletes the corresponding file from CacheDirectory.

Every time Lucene retires a file (each commit/merge produces a new segments_N and deletes the old ones), the local cached copy is orphaned on disk and never reclaimed.

Over time the cache directory grows without bound and fills the drive. On Azure App Service this exhausts C:\local\Temp.

Here is the contents of the CacheDirectory:
Image

While the contents on Azure Storage are:
Image

Steps to reproduce

  1. Create an AzureDirectory with a persistent FS cache: new AzureDirectory(blobServiceClient, "container/index", new SimpleFSDirectory(tempDir), true).
  2. Open an IndexWriter (OpenMode.CREATE_OR_APPEND), add/update a few documents, Commit(), and dispose.
  3. Repeat step 2 many times (e.g. in a loop, or a scheduled job committing every few seconds).
  4. Inspect tempDir.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions