Skip to content

[SPIKE] Combine ElasticSearch Filters with Folder-based Criteria #33241

Description

@fabrizzio-dotCMS

Spike Title

Combine ElasticSearch Filters with Folder-based Criteria

Research Question

Goal

Explore approaches to combine ElasticSearch filters with folder-based criteria so that the Content Drive can return only the expected results.

Open Questions
• How can we link folders to content in ES queries if folders are not indexed?
• Should folder information be included in the ES index for contentlets (e.g. store folder path/ID)?
• Alternatively, should we first filter by ES, then cross-filter by folder results from the API?
• What would be the performance impact of each approach?

Perhaps the desired result can be accomplished if we first request all the identifiers within a folder and feed our query as follows using the identifiers like this:

+systemType:false 
-deleted:true 
+working:true 
+identifier:(
  48190c8c-42c4-46af-8d1a-0cd5db894797 OR 
  9f8a3b22-3e4d-4a97-92c6-df2f76b7b3d9 OR 
  6c7d2f55-1d8f-463c-8fb1-9e5c42b1e2a1 OR 
  2b7f95d4-1c1b-4e6f-8f5a-2a6f1e6d7c3a OR 
  d3f9e2a8-7b4c-4f92-8e16-4b9f8f3a2e5d OR 
  a8b2c6d1-4f9e-42c8-9e1d-7f6b3a2d4c9e OR 
  3e9d7f1b-5a2c-4d8f-9e1b-6c7f2a9d8b1e OR 
  f2a7c9d8-3b1e-4a9d-8f2c-7b3d9e1f6a4c OR 
  7b3e1f9d-2c8a-4f7d-9e3b-1a6c9d2f8b5e OR 
  c9d2f8b5-7a3e-4f1b-9e6c-2d8f3a7b1e4c
) 
+title:some

This will fulfill the requirements

However, if the number of items within a folder is too long, the query might easily reach a limit

Timebox

8h

Acceptance Criteria

  • [] Provide a documented solution with the approach to follow
  • [] Test if the proposed solution of using identifiers Scales

Context

Currently:
• ElasticSearch allows us to filter by properties such as title, contentType, host, etc.
• Our folder API exposes the folder hierarchy, but ES does not store or index folder data.

This creates a challenge when we need queries like:

Retrieve all contentlets where the title contains "some" and belongs to the selected folder.

Links

No response

Activity

  1. moved this from New to Current Sprint Backlog in dotCMS - Product Planningon Sep 10, 2025
  2. zJaaal commented on Sep 16, 2025

    @zJaaal
    Member

    @fabrizzio-dotCMS Is it possible to add the full locale information and not only the languageId?

  3. fabrizzio-dotCMS commented on Sep 16, 2025

    @fabrizzio-dotCMS
    MemberAuthor

    @fabrizzio-dotCMS Is it possible to add the full locale information and not only the languageId?
    @zJaaal Let's see how it gets done first. I was told that BrowserResource fills the gaps between DB and ES. I need to do this research first to give you a concrete answer.

  4. moved this from Current Sprint Backlog to In Progress in dotCMS - Product Planningon Sep 16, 2025
  5. fabrizzio-dotCMS commented on Sep 16, 2025

    @fabrizzio-dotCMS
    MemberAuthor

    Well, it turns out that BrowserResource can actually be used to integrate search with folders.
    The catch is that it does not rely on ElasticSearch.

    On the other hand, it supports filtering by a bunch of criteria — name, extension, baseTypes, mimeTypes, languages — and it also respects user permissions.

    However, to track any potential side effects caused by the introduction of changes, I need to know where it is currently used
    I found this:

    /**
       * Retrieves contentlets from a specified folder.
       *
       * @param params - Parameters defining the folder and retrieval options.
       * @returns An observable emitting an array of `DotCMSContentlet` items.
       */
      getContentByFolder(params: ContentByFolderParams) {
          return this.#http
              .post<{ entity: { list: DotCMSContentlet[] } }>('/api/v1/browser', params)
              .pipe(pluck('entity', 'list'));
      }
    

    on core-web/libs/data-access/src/lib/dot-site/dot-site.service.ts
    @zJaaal can you help me track where that endpoint is being used in the FE?

    BTW I think I can add the language as a locale

  6. fabrizzio-dotCMS commented on Sep 16, 2025

    @fabrizzio-dotCMS
    MemberAuthor

    I just confirmed BrowserResource is only used by the File Selector
    #30216
    But the underlying API is consumed via BrowserAjax

  7. fabrizzio-dotCMS commented on Sep 17, 2025

    @fabrizzio-dotCMS
    MemberAuthor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions