Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
    Marty Schoch
    the reason is that good suggestions need to pull from dataset beyond just the term dictionary
    i think you could use the vellum project (also used by bleve) as the datstructure, you'd add things you want to suggest to it, and use the iterator to walk deeper into the vellum as more keystrokes come in
    but, im sure there is a lot more glue work there than i imagine
    and you have to figure out the lifecycle for updating it, when the data behind it changes
    @mschoch thnx for the answer :)

    @mschoch I'm trying to create keyword analyzer for text field mapping but I keep getting an exception "no analyzer with name or type 'keyword' registered". What am I doing wrong?

    indexMapping := bleve.NewIndexMapping()
    documentMapping := bleve.NewDocumentMapping()
    keywordFieldMapping := bleve.NewTextFieldMapping()
    keywordFieldMapping.Analyzer = keyword.Name
    documentMapping.AddFieldMappingsAt("fullVenueName", keywordFieldMapping)
    indexMapping.DefaultMapping = documentMapping

    I was following one of your presentations https://youtu.be/Vhc_1Ta3Mi0?t=1180

    Aman Gupta
    If I use bleve.New on master will that use Scorch?
    Marty Schoch
    @tmm1 no, the default is upsidedown.
    i have items indexed like "texting" "texted" "texts"
    i want do search "text"
    and result this ones
    how it could possible ?
    Marty Schoch
    @volknanebo there are a few different approaches
    typically one might choose to use an analyzer which "stems" words like "texting" "texted" and "texts" all into the single term "text" in the index
    then if you use a MatchQuery for "text" you'll find matches against all 3 of those original terms
    i just checked using this tool: http://analysis.blevesearch.com/analysis
    and if you use the "en" analyzer, it will do what i just described
    Aman Gupta
    i tried converting one of my indexes from leveldb to scorch and the results weren't so good
    i'm using leveldb with snappy for compression, so the scorch index was much bigger
    but there seems to be some sort of performance issue.. i was indexing 500 documents per batch with a total of about 30k documents.. as the index filled up each batch operation took longer and longer
    the first few sets of docs indexed in milliseconds, but towards the end each batch was taking 20s or more to index
    Stanislav Petr
    Hi, my question is about designing index structure... I have documents (files on filesystem) stored in bleve and i need to limit search only to some directory od directory with subdirectories. How to construct the index and query?
    i am new to this and found via search engine
    how to develop a simple search UI with highlighting and easy navigation using blevesearch similar to http://table.branham.org/#/main which has a) type-ahead b) no of matches found c) highlighting the search word d) easy navigation e) search only titles or content
    Ales Najmann

    hello folks, I'm indexing some archive and I'm facing something that I cannot grasp. In the archive I have a record with Author of type string and date of type number (it's year). And I was hoping for having queries that would select records in between interval. But if I use query like:

    +Author:Joe +Date:>=2016

    I end up with 2 results, but >= operator seams to be completely ignored, Joe has entries in 2017, 2018. I don't know exactly what's wrong with it.

    Matt Ouille
    I've seen some links on leveldb and whatnot, but how do I actually get bleve working with something like leveldb?
    Jonas Felix
    could i use bm25 score instead of tf / idf in bleve ?
    Fredrik Lönnblad

    hey, I have started looking at adding search to a few simple static web pages I have and wonder if there are any good examples for indexing, searching and highlighting HTML?

    currently when searching for go in <h3 id="go">go</h3><pre><code class="language-go">var i = 0</code></pre>, I get a highlight like this:

    <h3 id="<mark>go</mark>"><mark>go</mark></h3><pre><code class="language-<mark>go</mark>">var i = 0</code></pre>

    I would expect something more like this:

    <h3 id="go"><mark>go</mark></h3><pre><code class="language-go">var i = 0</code></pre>
    Onion Ltd
    hey, could anyone shine a little light on how to build a proper analyzer that returns results for a query "buzz" when matching document contains "BuzzFeed"? i understand that token filter camelcase is way to go but my experiments've brought no results. here's a gist of my index mapping: https://gist.github.com/onionltd/ff7970b1ccf8ee27f0b352599a4520cf
    Hey @mschoch (or anyone else who is responsible for snowballstem) - could use your help with blevesearch/snowballstem#2 - the repo is missing a LICENSE, i can't import it until it has one (legal restrictions and whatnot)
    Amrit Panesar
    greets everyone
    I'm running into an issue compiling the example code
    getting an err: panic: invalid page type: 0: 4
    this is triggered here:
            // open a new index
            mapping := bleve.NewIndexMapping()
            index, err := bleve.New("example.bleve", mapping)
            if err != nil {
    was expecting this to "Just Work (tm)"
    Amrit Panesar
    which actually bubbles up from:
    goroutine 1 [running]:
    github.com/etcd-io/bbolt.(*Cursor).search(0xc000145bb0, 0xc000145c10, 0x5, 0x20, 0x4)
            /home/neo/go/src/github.com/etcd-io/bbolt/cursor.go:250 +0x353
    github.com/etcd-io/bbolt.(*Cursor).seek(0xc000145bb0, 0xc000145c10, 0x5, 0x20, 0x0, 0x0, 0x0, 0x1, 0xc00029e000, 0x0, ...)
            /home/neo/go/src/github.com/etcd-io/bbolt/cursor.go:159 +0x7d
    github.com/etcd-io/bbolt.(*Bucket).Bucket(0xc000298018, 0xc000145c10, 0x5, 0x20, 0x5)
            /home/neo/go/src/github.com/etcd-io/bbolt/bucket.go:105 +0xd4
    github.com/blevesearch/bleve/index/store/boltdb.(*Store).Reader(0xc000296000, 0xe2bd80, 0xc000081ef0, 0xa6a860, 0xc000296000)
            /home/neo/go/src/github.com/blevesearch/bleve/index/store/boltdb/store.go:123 +0xac
    github.com/blevesearch/bleve/index/upsidedown.(*UpsideDownCouch).Open(0xc00013c380, 0x0, 0x0)
            /home/neo/go/src/github.com/blevesearch/bleve/index/upsidedown/upsidedown.go:315 +0x15f
    github.com/blevesearch/bleve.newIndexUsing(0x9ba0bf, 0xd, 0xa6eda0, 0xc00013c300, 0x9b970a, 0xb, 0x9b7c08, 0x6, 0x0, 0x0, ...)
            /home/neo/go/src/github.com/blevesearch/bleve/index_impl.go:109 +0x4ab
            /home/neo/karaokefiles-server/main.go:12 +0x98
    exit status 2
    Which is the best tool for generating javadoc in different formats ??? I want to generate javadoc with search engine.....Please advise...
    Frédéric G. MARAND
    Hello. I have a problem with some data: when the queried string appears several times in a document, I only get at most 1 fragment per field on matching documents. Others hits are not returned. My documents look like struct { Title string, Lines []string }, so I get at most 1 fragment on Title and 1 on Lines (assuming they match the query). How can I get all hits ?
    Kevin Langhans
    Hello everybody
    Does anyone know the max filesize for a bleve index stored in boltdb ?
    Marlon Hendred
    so is bleve the lucene equivelent written in go? is there an bleve backed elasticsearch equvilent written in go?
    Kevin Langhans
    Question #1 kind of
    Kevin Langhans
    Question #2 you have to be more specific “bleve backed elastic search equivalent...” what do you mean by equivalent?
    Marlon Hendred
    for question #2 i mean like a standalone search engine that you could index data into and search/filter via REST apis that hadles index replication/sharding etc for you
    Kevin Langhans
    As far as i have worked with bleve that is possible.
    I specifically chose bleve over elasticsearch because of the way lower memory requirements it has.
    I simply wrote asimple REST API in front of bleve and thats it.
    In regards of the sharding, i am nowhere near any requirements to shard right now, so i cant talk about that.
    Marlon Hendred
    @klanghans how big is the data set (doc count, disk space)? what is your read to write ratio to your bleve search app. what is the usecase?
    Kevin Langhans
    @msquaredh documents: 5000, 90/10 read/write, disc space: 24MB, this is the setup for one customer, multiply that by 1000.
    Each customer has a confined userspace set with a separate bleve index on its own.
    @msquaredh we found that with such a small footprint, bleve is ideal to isolate userspaces without having to use containers, which removes all the container management hassle and overhead from the project
    the use case: learning video producers use our product to manage their customers to upload/edit videos to a video platform
    @msquaredh one thing we found is that bleve really needs some good i/o performance otherwise it stresses the device bus by a lot