Typesense: ObjectNotFound error when there are no models yet

[gdebrauwer]

Scout Version

10.13.1

Scout Driver

Typesense

Laravel Version

11.44.0

PHP Version

8.4

Database Driver & Version

No response

SDK Version

No response

Meilisearch CLI Version

No response

Description

I have a model that I made searchable. The table of that model is still empty. If I run php artisan scout:import, then nothing happens because there are no models to import in Typesense.

The issue starts to occur when you try to search using Scout while there are still no models yet. Typesense throws an ObjectNotFound error because it can not find a Typesense collection for that model class. This is caused by changes of #898. It does not make sense that Scout throws an exception when searching just because there are no models yet.

Steps To Reproduce

• make a model searchable but don't create any objects
• run php artisan scout:import
• run YourModel::search('foo')->get() -> You will get an ObjectNotFound typesense exception
• create a model that is indexed by Scout
• run YourModel::search('foo')->get() again -> You don't get an error anymore

[github-actions]

Thank you for reporting this issue!

As Laravel is an open source project, we rely on the community to help us diagnose and fix issues as it is not possible to research and fix every issue reported to us via GitHub.

If possible, please make a pull request fixing the issue you have described, along with corresponding tests. All pull requests are promptly reviewed by the Laravel team.

Thank you!

[tharropoulos]

Hey there, the reason for that behavior is that the getOrCreateCollection function will first be fired when indexing a document. Otherwise, the reserved name for the collection (the model's name), will return that error, since the resource does not exist.

I'm not sure what else should be done here, would you prefer a different kind of error to be thrown? Checking for the existence of the collection shouldn't be done when searching, it will bog down search performance with another API call

[gdebrauwer]

When the php artisan scout:import command is run, can we not create an empty collection even if the table of the model you want to index is still empty? 🤔

[tharropoulos]

Sure, I'll have to check the flow and how it's handled though and report back

[tharropoulos]

Dug some more into it:

When you run php artisan scout:import on a model with an empty table, here's what happens:

1. The import command instantiates your model and calls makeAllSearchable() [1]
2. makeAllSearchable() builds a query and calls the searchable() macro on the query builder [1][2][3][4]
3. The searchable() macro uses chunkById() to process records in batches [1] [2]
4. Since there are no records, orderedChunkById() exits early [1] with:

   if ($countResults == 0) {
       // No results, break out of processing loop
       break;
   }

The key issue is that the callback function containing $models->filter->shouldBeSearchable()->searchable() never executes because there are no chunks to process. This means no collection creation code runs.

Prior to PR #898, Typesense was checking for (and creating if needed) the collection on both indexing AND searching operations. Now collections are only created during indexing, but with empty tables, that code path isn't reached.

[gdebrauwer]

@tharropoulos Could we maybe add Typesense support to https://github.com/laravel/scout/blob/10.x/src/Console/IndexCommand.php or https://github.com/laravel/scout/blob/10.x/src/Console/SyncIndexSettingsCommand.php so that it is possible to create the necessary typesense collections via an artisan command? 🤔

[tharropoulos]

Sure, if @jasonbosco and @taylorotwell are down for that, since this will change the public-facing API

[tharropoulos]

I'm not happy about allowing users to create indexes manually as it may create confusion. Previously, the indexes were created automatically (and still are) if there were any documents to be indexed. Adding another command that essentially creates two branches of creating collections isn't something I'm really for.
Instead, we could catch the ObjectNotFound exception from Typesense during search and throw a more descriptive exception with a helpful message explaining that there haven't been any documents to be imported in the collection, and to create it you need to use the import command with some documents to import.
This maintains our existing architecture while providing better guidance to developers without introducing new ways to create collections.

[andyleach]

I'm not happy about allowing users to create indexes manually as it may create confusion. Previously, the indexes were created automatically (and still are) if there were any documents to be indexed. Adding another command that essentially creates two branches of creating collections isn't something I'm really for. Instead, we could catch the ObjectNotFound exception from Typesense during search and throw a more descriptive exception with a helpful message explaining that there haven't been any documents to be imported in the collection, and to create it you need to use the import command with some documents to import. This maintains our existing architecture while providing better guidance to developers without introducing new ways to create collections.

My two cents is that there is a gap here. I can see two different paths forward that maintain what you desire:

• Attempt to create the index on attempting a search if the collection doesn't exist
• On search where the collection doesn't exist, return that there are no results as an empty array or whatever the ScoutBuilder expects.

The lack of data shouldn't be the determining factor on our ability to perform a search.

If skipping the presence check is the desire, you could naively perform the search, and on a ObjectNotFound exception, you could then create the index and return no results.

[tharropoulos]

My two cents is that there is a gap here. I can see two different paths forward that maintain what you desire:

* Attempt to create the index on attempting a search if the collection doesn't exist

* On search where the collection doesn't exist, return that there are no results as an empty array or whatever the ScoutBuilder expects.

The lack of data shouldn't be the determining factor on our ability to perform a search.

If skipping the presence check is the desire, you could naively perform the search, and on a ObjectNotFound exception, you could then create the index and return no results.

I'd prefer actually creating it on that error, instead of returning a mocked response, as the Typesense API may change in the future. #917 Does this fall line with what you were describing?

[andyleach]

My two cents is that there is a gap here. I can see two different paths forward that maintain what you desire:

* Attempt to create the index on attempting a search if the collection doesn't exist

* On search where the collection doesn't exist, return that there are no results as an empty array or whatever the ScoutBuilder expects.

The lack of data shouldn't be the determining factor on our ability to perform a search.
If skipping the presence check is the desire, you could naively perform the search, and on a ObjectNotFound exception, you could then create the index and return no results.

I'd prefer actually creating it on that error, instead of returning a mocked response, as the Typesense API may change in the future. #917 Does this fall line with what you were describing?

It does

[tharropoulos]

It does

Nice, I'll open it up for review tomorrow

[tharropoulos]

It should now create the collection if it's not found when searching in it.

@gdebrauwer could you try checking it out as well to verify that it solves this issue?