I've been following the project for a while now and love seeing how active the developer community is behind the project. I noticed you're using Lucene.NET for the indexing, with which I have no knowledge of. I am wondering though, why you chose Lucene.NET and not something like elastic search. Are you trying to keep SenseNet and it's features self contained? IE no external dependencies requiring setup and configuration/administration?
Hi @krptodr, thank you for stopping by :). TLDR: we plan to look into the search service subject mid-term, elastic is one of the candidates, yes :). Early on we started the project with SQL Server search. However it became clear that it is not flexible enough for us - for example it was not possible (or feasible) to build column indexes for a random combination of fields in a query. We moved to Lucene because it allowed us to use its query API directly (in the application process, without network overhead), which means it was very fast, regardless of the fields that you put into the query. But this architecture also came with a drawback: we had to maintain indexes locally, on every web node, and notify other nodes every time the index has changed to synchronize indexes. This worked well in an old-school "IIS+NetFramework on a VM" environment, but in a modern, containerized environment would be hard to maintain and automatize this architecture. And this is how we arrived to the current, service-based search architecture. Currently there is a single, centralized search service, accessed by all the web nodes. This was the first step toward a really scalable search service - because currently it is not possible to scale up this service: Lucene (at least the version we use) does not allow that. The next step would be one of the following options:
upgrade to the latest Lucene.Net version if it supports sharding/partitioning (I believe it does)
implement a different search service, based on a similar technology (ElasticSearch, Solr, etc) The architectural issues we face:
we have a flexible type system which lets users add fields dynamically, while the system is running. Not every search API allows that (without rebuilding the index).
we need to filter the search results by permissions before they are returned to the web app . It is not an option to let millions of result ids flow through the network every time and filter the results on the web server, it would be too slow. Let us know if you have more questions or suggestions :).
Hi sensenet Team, while searching for a headless cms i come across this platform and found quite interesting in the concept. We had already evaluated couple of other platforms like contentful and dotcms etc. But none of then matched our requirement. Before proceeding i would like to ask couple of question which may seems childsh.
I had seen content delivery api and content management api in all other platforms, but here it is quite different, i guessed it is retrieved through a query engine. But what if if i need to use cdn for content delivery to get it faster globally? what is the solution for that in sensenet. It will be better if i can know advatnages of query based content retrieval vs API based delivery.
Second point was, regarding events and communication, we are looking to have a real time discusion wall in front end for each workspaces, can i achieve this with sensenet? how do you manage real time communication with sensenet?
Third point is regarding Workflow, we like to automate various process like approval of purchase order, etc in front end with workflow that supports bpmn2. I never saw any details about workflow, do you have plans for implementing workflow engine or connectors for anything like Activity or processmaker or Microsoft Power Automate?
Hi @jamesoliver100 , let me answer you questions here briefly and if it is ok for you, we can continue discussing your needs and questions through email. Feel free to contact us on email@example.com.
Comparing query based content retrieval delivery with API based the important difference is that the query based one is far more flexible with the possibility of filtering by path or type, selecting fields, expanding references, etc. To explore the capabilities I would recommend looking at the new documentation at https://docs.sensenet.com/api-docs/basic-concepts. We do not have a CDN solution now but we’re working constantly to make sensenet available to be installed and run on the popular cloud services, which I think would be an answer to this question.
In sensenet we do not have any solution for real time communication by default, this kind of features can be added with custom development or with integrating a third party solution, but to give you an accurate answer I need further information about your business needs.
Since sensenet is now moved to .net core, the old workflow solution with Microsoft Workflow Foundation is not available anymore. We’re working now to implement webhooks which we hope to make it available create workflowish things. As I mentioned above we’re eager to hear about your business needs and suggestions, please drop us a mail to firstname.lastname@example.org.
@herflis thanks for the reply. I understood the difference. So you mean we can use any front end language with same query language to retrieve? then it will be great. for the other things i shall shoot up a mail
@jamesoliver100 theoretically, that is correct. We are looking forward to your mail and hearing from the other things :)