Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Aug 13 06:13
    CognitiveClouds-Prasad commented #1
  • Aug 13 06:12
    CognitiveClouds-Prasad commented #1
  • Aug 12 15:00
    fergiemcdowall commented #1
  • Aug 12 14:35
    CognitiveClouds-Prasad commented #1
  • Aug 12 14:33
    CognitiveClouds-Prasad commented #1
  • Aug 12 14:13
    fergiemcdowall commented #1
  • Aug 08 11:49
    eklem commented #58
  • Aug 08 11:46
    CognitiveClouds-Prasad opened #1
  • Aug 05 12:40
    fergiemcdowall commented #490
  • Aug 05 11:58
    eklem commented #490
  • Aug 05 11:55
    eklem opened #490
  • Aug 05 11:53

    eklem on master

    Doesn't work on nodejs v12 Fai… (compare)

  • Aug 05 11:41

    eklem on master

    Added node.js v12 testing On T… (compare)

  • Aug 05 11:38

    eklem on master

    Changed tests from nodejs v11 t… (compare)

  • Jul 25 18:17
    eklem closed #75
  • Jul 25 18:17
    eklem commented #75
  • Jul 25 18:06

    eklem on v0.3.3

    (compare)

  • Jul 25 17:56

    eklem on master

    renaming lib to sw in dist (compare)

  • Jul 25 17:52

    eklem on master

    Production-mode in webpac.confi… (compare)

  • Jul 25 17:49

    eklem on master

    demo pointing to .latest.js (compare)

Fergus McDowall
@fergiemcdowall
Would be really interested in getting a look at your setup
Jonathan Poltak Samosir
@poltak

@fergiemcdowall Not sure exactly what would be of interest, so here's a gist (non-runnable) containing an example doc that gets indexed, simple query with a filter, a more realistic (in relation to use within the program) query with a filter, an example scoringCriteria object from a search result with a NaN score, plus the index/field options being used:
https://gist.github.com/poltak/6c8c003ea8a59407be5b929e2002e418

Bit of a mess, so let me know if you need clarification on any parts or the reasonings for certain choices (and no rush looking at this; everyone has stuff to do). As I was putting it together, line 89 in the scoringCriteria dump stood out to me - not sure if there should be a null value there.

Fergus McDowall
@fergiemcdowall

In order to fix an indexing/search bug the project needs:

1) a simple script that reproduces the error by indexing one or more documents and then running a query (dont have to bother with a test framework- just dump to console.log)
2) a note on the expected vs actual result.

or
a very specific error report: "indexing documents containing the word 'avacado' crashes my system"
But basically- if you provide a runnable script, it will always get fixed
Jonathan Poltak Samosir
@poltak
Totally understood. Sadly I don't have much time to do anything lately, but will try to get a runnable script to you in the next few weeks if I can't resolve it trying other things in the meantime. Got some ideas on ways I could do the queries differently to see how the results differ
Jonathan Poltak Samosir
@poltak

@fergiemcdowall after encountering a much more severe issue regarding unexpected results (using offset + pageSize pagination query opts) with our index implementation, and being completely lost with it, I wrote up a node script this morning to try and reproduce it. Sadly I haven't been able to reproduce this one in node (yet). As I already had the script working I instead decided to try and get it to reproduce those NaN scores on filter search I was talking about last week.

While in a lot of queries I (frustratingly) cannot get it to happen as it is happening in our browser implementation, however I did found a query that manages to return all the expected results but with NaN scores.
Here's the gist that should be runnable in >=Node 7.6:
https://gist.github.com/poltak/7eb8128d9bbc6d784a38a922f8d8560c

No rush having a look. Some simple instructions up there, but let me know if you need any clarification on what or why I'm doing things - I expect it's something I'm doing wrong. The data is taken directly from our app, although note that I'm cutting out a lot of the content field as it's not really important for the purposes here (and can be quite long). The index opts are as our index is set up.

Fergus McDowall
@fergiemcdowall
great- will take a look
have opened an issue here: fergiemcdowall/search-index#413
Jonathan Poltak Samosir
@poltak
Thanks @fergiemcdowall. Appreciate it.
swissums
@swissums
I would like to see the index-time of individual words as they are indexed. I was thinking just a simple console.time(), Where is the best place to add that into search-index?
Espen Klem
@eklem
@swissums Somewhere in search-index-adder ?
Espen Klem
@eklem
I don't know the add-to-index part well enough, but it's a place to start looking.
Fergus McDowall
@fergiemcdowall
Thats a good question @swissums
There's a few ways to do that
swissums
@swissums
@fergiemcdowall Yeah I managed to fit it in there quite nicely thanks for the help =)
Espen Klem
@eklem
:+1:
Espen Klem
@eklem
@fergiemcdowall is the /dist folder the browserified version of search-index?
Fergus McDowall
@fergiemcdowall
@eklem yes!
Espen Klem
@eklem
Thank you =) I'll start making norch-vue-app ready for truly running in the browser.
Espen Klem
@eklem
I think I've done the same error with norch-vue-app as you did with norch-angular-app, making it too slick/too much overhead. I'll try to make a simpler integration between norch/search-index and Vuejs. What I need is support for all the endpoints/functions of norch/search-index
Fergus McDowall
@fergiemcdowall
@eklem easily done! :)
Espen Klem
@eklem
Yes, wanted to do two tasks in one go. Make a framework everybody could use AND make it look good.
Ducros Alix
@Karalix
Hi, I have a question about the difference between add() and concurrentAdd() method : in most examples, before using add(), si.defaultPipeline(batchOptions) is called. If I want to use concurrentAdd() instead of add(), do I still need to do a similar call ? How could I if I have not a stream ?
Working on 0.9.16 if it can be of any help
Fergus McDowall
@fergiemcdowall
Hi Ducros! Yes, you have to use a pipeline- it turns the data into a digestible format
.concurrentAdd may work without
There are some examples in the tests
Ducros Alix
@Karalix
Thanks :smile:
Mustafa Hanif
@mustafa-hanif
how to load index from file
Espen Klem
@eklem
Have you exported an index, @mustafa-hanif? And is it norch or search-index you're working with?
Espen Klem
@eklem
I think we need to understand a little what you are trying to achieve and how. Any code you can show us, or an explanation of your setup?
Espen Klem
@eklem
From GitHub: request for comment - move to promises-based API? fergiemcdowall/search-index#446
Espen Klem
@eklem
How is the move going, @fergiemcdowall ?
Fergus McDowall
@fergiemcdowall
Making progress. Doing it in a private repo
Vladimir Vujosevic
@frontmesh
Hi everyone, I have a question regarding search-index usage. I have a large collection in MongoDB with over 2 million documents. Performing a regex query can take up from 1s to over 10s in some cases. Is it possible to improve speed of these queries by using search-index? is this a use case for this library at all ?
Espen Klem
@eklem
Yes. I think it was tested with 1 or 5 million documents, but got rewritten to have a streaming interface to work better with large document sets. I'm not sure what will be the actual response time, but I'm guessing it's quite a lot better.
Vladimir Vujosevic
@frontmesh
Thanks @eklem. Is there maybe any example on how to connect it with mongodb collection and read/query/search data from it ?
Fergus McDowall
@fergiemcdowall
@frontmesh no documentation around that Im afraid
Vladimir Vujosevic
@frontmesh
@fergiemcdowall Thanks. I'll see if i can come up with something
Espen Klem
@eklem
Just a heads up @all: There's a big rewrite of search-index, and breaking change, as of v1.0.0. New Promise API and you can chain and/or/not in a much more flexible way.
Espen Klem
@eklem
Browser example to come soon
prog20901
@prog20901
I am very happy to come across this group
Looking for a similar app or portable solution like below - type-ahead search with easy navigation, number of hits - ex: http://table.branham.org/#/main - where huge pre-defined text has been dumped for easy query and navigation
Is it possible via search-index? can someone help me with a demo sample or good example to build like the above
everything is very quick and fast and search results in less than a second
Fergus McDowall
@fergiemcdowall
prog20901
@prog20901
@fergiemcdowall Can you please help me with some UI examples so that will see how i can bring up like above? for me it looks complex. any demo or sample start would be helpful