@msquaredh we found that with such a small footprint, bleve is ideal to isolate userspaces without having to use containers, which removes all the container management hassle and overhead from the project
the use case: learning video producers use our product to manage their customers to upload/edit videos to a video platform
@msquaredh one thing we found is that bleve really needs some good i/o performance otherwise it stresses the device bus by a lot
Hi all, I'd like to ask whether the current version of bleve has an autocomplete feature? I saw some open issues regarding this: blevesearch/bleve#858 blevesearch/bleve#377 and some discussion where was suggested that the core have to be changed to address this functionality: https://groups.google.com/forum/#!topic/bleve/zd6vWSuCIEA
If it isn't possible, what's the current state of affairs, is this on roadmap?
If it isn't possible, what's the current state of affairs, is this on roadmap?
I would like to index accounts and groups. The struct is generated using protobuf, but I can add the
func BleveType() string to them. How can I add a type to the indexed document, so that I can build queries to only list accounts or groups? Or should I be using two indexes for that? Ideally I can later query for accounts and groups and get a mixed set of results, but for that I need to have a property in the index ... but how if I cannot add it to the protobuf?
Hm, I could wrap the struct in my own struct ... with an additional type property ...
Hi All, could anyone help me with the Html CharFilter? I was trying to write my own Analyzer + html CharFilter, but it seems the html tags did not being stripe out:
https://repl.it/@larrypage/FarawayKindheartedStartup#main.go
https://repl.it/@larrypage/FarawayKindheartedStartup#main.go
I created my own Analyzer just directly copy the "en" analyzer, and also on top of that I added the html CharFilter
@mandeepkhatry today you have to choose, whether you want to index age as text or number field (in the index), and that choice should be made based on the types of queries you want to support. if you need to do numeric range searches, you should index the field as a number. if you only need exact matches, you should index the field as text
now, that just relates to how you set up your mapping, after making that choice, it will limit which types of input get handled
if you choose text, only the string version would get indexed, if you choose number, only the numeric value would get indexed
we're actually in the middle of discussing some aproaches to allow indexing numeric values as text, but today it is not possible, you would have to "fixup" your data manually before passing it to bleve
@mandeepkhatry second question, no scorch is not the default, but it is what we recommend
not sure about that last question, yes if you indexed age as a numeric field, you would do a numeric range query with min and max set to 10
and call SetField on the query to "age"
for these types of questions, it's often best to share a 20 line program that we can discuss the behavior of
@mschoch I have a code https://play.golang.org/p/Iq6YfMF7LGk
When I run my code, I get an error
start batch indexing....
panic: runtime error: index out of range [-1]
goroutine 25 [running]:
github.com/blevesearch/zap/v11.mergeStoredAndRemap.func2(0xc001483130, 0x9, 0x74, 0xc01181aeab, 0xc, 0xc, 0x0, 0x0, 0x0, 0x1)
/home/mandeep/work/pkg/mod/github.com/blevesearch/zap/v11@v11.0.9/merge.go:687 +0x579
github.com/blevesearch/zap/v11.(*SegmentBase).visitDocument(0xc0083ca000, 0xc00041baa0, 0x1287, 0xc00038f618, 0x19, 0x0)
/home/mandeep/work/pkg/mod/github.com/blevesearch/zap/v11@v11.0.9/segment.go:384 +0x383
github.com/blevesearch/zap/v11.mergeStoredAndRemap(0xc007a80c80, 0xa, 0xa, 0xc007a80c30, 0xa, 0xa, 0xc004103380, 0xc016700000, 0x271a2, 0x271a2, ...)
/home/mandeep/work/pkg/mod/github.com/blevesearch/zap/v11@v11.0.9/merge.go:685 +0x47f
github.com/blevesearch/zap/v11.MergeToWriter(0xc007a80c80, 0xa, 0xa, 0xc007a80c30, 0xa, 0xa, 0x400, 0xc004103350, 0xc00c0cd5c0, 0x203001, ...)
/home/mandeep/work/pkg/mod/github.com/blevesearch/zap/v11@v11.0.9/merge.go:132 +0x1ef
github.com/blevesearch/zap/v11.mergeSegmentBases(0xc007a80c80, 0xa, 0xa, 0xc007a80c30, 0xa, 0xa, 0xc013fcfe90, 0x22, 0xc000000400, 0xc00c0cd5c0, ...)
/home/mandeep/work/pkg/mod/github.com/blevesearch/zap/v11@v11.0.9/merge.go:79 +0x24c
github.com/blevesearch/zap/v11.(*ZapPlugin).Merge(0xfd0548, 0xc013f9ae60, 0xa, 0xa, 0xc007a80c30, 0xa, 0xa, 0xc013fcfe90, 0x22, 0xc00c0cd5c0, ...)
/home/mandeep/work/pkg/mod/github.com/blevesearch/zap/v11@v11.0.9/merge.go:54 +0x14f
github.com/blevesearch/bleve/index/scorch.(*Scorch).planMergeAtSnapshot(0xc0002c2000, 0xb68640, 0xc0000c2040, 0xc00030a000, 0xc00e8d4e00, 0x0, 0x0)
/home/mandeep/work/pkg/mod/github.com/blevesearch/bleve@v1.0.9/index/scorch/merge.go:321 +0xad9
github.com/blevesearch/bleve/index/scorch.(*Scorch).mergerLoop(0xc0002c2000)
/home/mandeep/work/pkg/mod/github.com/blevesearch/bleve@v1.0.9/index/scorch/merge.go:68 +0x2c7
created by github.com/blevesearch/bleve/index/scorch.(*Scorch).Open
/home/mandeep/work/pkg/mod/github.com/blevesearch/bleve@v1.0.9/index/scorch/scorch.go:206 +0x125
exit status 2Any problem on this simple code?
@mandeepkhatry I don't see any obvious problem, reproduced the issue. I have opened an issue to track it here: blevesearch/zap#28
I tried with document {"name350349" : "TestName350439", "age350349" : 350449"} only single document (without loop testing this particular document only), it works fine.
there is currently no way around this, the file format uses a fixed-width 16-bit integer for field identifier, and all the surrounding code makes the same assumption
because of the way you're using fields unique to documents, the problem only shows up when we merge 2 segments, and the resulting segment would require > 2^16 fields in the same segment
i suspect that is why it may work in the most recent example you gave, even though you used more fields than allowed, it doesn't actually crash until a merge happens
@mschoch Thanks for being so active in the community.
I have a data with field "age" : "22" 22 as string with ID 1 and another data with field "age" : 22 as number with ID 2.
My searching
bleve.NewQueryStringQuery(age:22)
I get ids both [1,2]
How can I use query to query only number age and only string age?
Also seperate between string "false" and boolean false in QueryStringQuery
@mschoch
For matching queries related to numeric to match data with integer age 22
bleve.NewQueryStringQuery("age:>=22 age:<=22") seems to work
FOr string to match data with string age "22"
bleve.NewQueryStringQuery("age:'22' ")
What do u think on this?
Or is there anything something like for boolean we use isEmployee:T* . Is there any anything for numeric without doing bleve.NewQueryStringQuery("age:>=22 age:<=22") ?
@mschoch Can i use NewMatchQuery for search date and time ?
there was no need to send it 3 times, first time senders required approval
Hello, I expect I'm making some beginner mistakes here with the mappers... can someone take a look at this?
type Note struct {
id string
title string
blocks []BasicBlock
}
func (n Note) Type() string {
return "note"
}
type BasicBlock struct {
id string
contents string
}
func (b BasicBlock) Type() string {
return "block"
}
englishMapping := bleve.NewTextFieldMapping()
englishMapping.Analyzer = "en"
blockMapping := bleve.NewDocumentMapping()
blockMapping.AddFieldMappingsAt("contents", englishMapping)
noteMapping := bleve.NewDocumentMapping()
noteMapping.AddFieldMappingsAt("title", englishMapping)
noteMapping.AddSubDocumentMapping("blocks", blockMapping)
indexMapping := bleve.NewIndexMapping()
indexMapping.AddDocumentMapping(Note{}.Type(), noteMapping)All queries return zero results except MatchAllQueries. Ideas?
an even more minimal example:
index.Index("note1.org", Note{id: "note1.org", title: "note1.org"})
query := bleve.NewMatchQuery("note1.org")
search := bleve.NewSearchRequest(query)
searchResults, err := index.Search(search)