Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jul 25 17:47
    fergiemcdowall commented #586
  • Jul 25 17:45

    fergiemcdowall on v3.2.0

    (compare)

  • Jul 25 17:42

    fergiemcdowall on master

    bump node test versions version bump (compare)

  • Jul 25 16:19
    mikaelkaron commented #586
  • Jul 25 13:06
    fergiemcdowall closed #585
  • Jul 25 13:06

    fergiemcdowall on master

    feat: switch to `abstract-level` fix: install `fergies-inverted-… fix: fix require path for `si` and 4 more (compare)

  • Jul 25 13:06
    fergiemcdowall closed #586
  • Jul 25 13:06
    fergiemcdowall commented #586
  • Jul 25 12:45
    mikaelkaron commented #586
  • Jul 25 12:44
    mikaelkaron review_requested #586
  • Jul 25 12:40
    mikaelkaron synchronize #586
  • Jul 25 12:28
    mikaelkaron synchronize #586
  • Jul 23 01:38
    dependabot[bot] labeled #588
  • Jul 23 01:38
    dependabot[bot] opened #588
  • Jul 23 01:38

    dependabot[bot] on npm_and_yarn

    Bump file-type from 16.5.3 to 1… (compare)

  • Jul 21 20:02

    eklem on npm_and_yarn

    (compare)

  • Jul 21 20:02

    eklem on master

    Bump terser from 5.14.1 to 5.14… Merge pull request #587 from fe… (compare)

  • Jul 21 20:02
    eklem closed #587
  • Jul 21 07:09
    dependabot[bot] labeled #587
  • Jul 21 07:09
    dependabot[bot] opened #587
Jeff See
@jeffsee55
I saw that! Seems like it's close, is the reason array values don't get stripped of special characters for something specific?
Fergus McDowall
@fergiemcdowall
Yes- its so that you can easily do your own tokenization and use non-ascii chars
So for instance- I work a lot with scandinavian languages and sentences like "bøker er gøy" are not always tokenized correctly
(due to the non-ascii ø)
Jeff See
@jeffsee55
Ok, so if I had an item like:
  const data = [
    {
      name: 'Homer Simpson',
      comments: ['doh', 'this is another comment'],
      _id: 'content/authors/homer.md'
    }
  ]
Fergus McDowall
@fergiemcdowall
therefore its easier to do ['bøker', 'er', 'gøy']
You could also introduce ngrams: ['bøker er', 'er gøy']
Jeff See
@jeffsee55

Would I get a hit for:

  await db
    .QUERY({
      FIELD: ['comments'],
      VALUE: 'another'
    })
    .then(print

Or does that need to be indexed manually by me

Fergus McDowall
@fergiemcdowall
ëtc.
lets see...
no
Jeff See
@jeffsee55
Ok that's what I'm seeing too I just wasn't sure if I was doing something wrong
Fergus McDowall
@fergiemcdowall
 const data = [
    {
      name: 'Homer Simpson',
      comments: 'doh this is another comment',
      _id: 'content/authors/homer.md'
    }
  ]
This would tokenize comments to 'doh', 'this', 'is', 'another', 'comment'
and you would then get a hit for VALUE: 'another'
comments: ['doh', 'this is another comment'] would allow you to search for VALUE: 'doh' and VALUE: 'this is another comment'
Jeff See
@jeffsee55
Ok, thanks for the explanation. I'm trying to see if I can use this for full search of potentially complex objects. Some might have an array of objects - which I'd like to sort of "flatten" out into something that gets indexed in the normal way
But maybe what I'm starting to understand is that there's no reason to keep those values as arrays, I can just merge them as you've done in the last example into a single string
Fergus McDowall
@fergiemcdowall
Yes, for american english, that is probably the easiest
If you want
to preserve spesial chars and punctuation, then arrays are the way to go
Jeff See
@jeffsee55

to preserve spesial chars and punctuation, then arrays are the way to go

But using arrays will also remove the ability to search for partial matches, correct? And to support those I'd have to use ngrams

Fergus McDowall
@fergiemcdowall
yes, you could also look into DICTIONARY
Nice to talk to you Jeff! I need to pick up my kid from kindergarten now, but I can get back to you later if you have any more questions
Jeff See
@jeffsee55
Thanks @fergiemcdowall for the help, I'll see if I can learn more about DICTIONARY
brightinnovator
@brightinnovator
I have a question on javascript. can someone help me on the javascript issue?
brightinnovator
@brightinnovator

I want to read 3 crore csv rows which is of 2GB csv file size and need to insert into MySQL via Java.

Could someone please help me know the fastest and memory efficient way to avoid out of memory exception as well load in lesser time?

Please kindly advise.

goodev2021
@goodev2021
how to develop an full content text search application using search-index with watermelon db (https://watermelondb.now.sh/)?
Fergus McDowall
@fergiemcdowall
You would need to find a leveldown that works for watermelondb
You would need to find a leveldown that works for watermelondb