From Elastic Search to Orama - is it possibile and suggested?

[VittorioMorellini]

Scenario:
A portal that manages informative articles, articles written about some areas like: work, fiscal, company...., is written in Next.js 12 (typescript) and the articles are on Sql Server table.
Because the search of articles in Sql Server was slow, we decided to implement Elastic Search
There is a batch C# that syncronizes an index on Elastic search with all the articles (or post) and their informations necessary for the research
The size of articles loaded are little more than 104.000
The search of articles from Next works on the index of Elastic search. We use in next the dependency: "@elastic/elasticsearch"
I follow you Michele since the first time you presented your Next.js book: The real word.

Question:
Is it suggested in this scenario to migrate for us from Elastic to Orama for the search?
Can Orama be filled offline with a Batch or some else procedure?
I read a few examples and I understood it is loaded in memory.
I did not understand if Orama is both a Database in memory and a DB persisted with an index on DB, or is mainly a memory database for the research.
Orama has to be filled in memory every time the app starts? how is done the syncronization with an external Database that is the main database for the on going application?
Sorry for my questions, I just want to understand better how Orama works, you can suggest me a documentation to improve in Orama Architecture

Best Regards
Vittorio Morellini

[mariocampacci]

@VittorioMorellini from my experience Orama shouldn't be used with very big indexes. Orama stores everything in memory and this can lead to a lot of RAM usage. It depends on the document structure but for 100k records the RAM usage could go up to 400MB.
When indexing million of records i find myself managing GBs of RAM usage...
Seems not ideal to me to use that amount of memory so I found an alternative that uses RAM only for index and saves the rest on the disk: meilisearch does the trick for me!

But I'm happy to listen to other thoughts

[zereraz]

Can documentsStore be used so that we won't have to load all the documents in RAM?