strongloop-community/loopback-connector-elastic-search

Strongloop Loopback connector for Elasticsearch

Bharath Reddy Kontham

@bharathkontham

An overview on updates in https://github.com/bharathkontham/loopback-connector-elastic-search

Only Elasticsearch 6.x is supported, ES 5.x or older version may or may not work.
Currently only one Index can be configured in one ES datasource of loopback (working on multiple indices configuration support) but we can use same datasource multiple loopback models.
A keyword field "docType" is used to store model names which provides support to use same Mapping Type of a single index for storing multiple models. This is major change in connector as earlier version used different mapping types to store different loopback models.
Nested objects querying (array of objects type in loopback), geo spatial queries and like, nlike are supported.

I'll update examples and documentation soon.

I'm working only on ElasticSearch 6.x or later support (no intention to work on older versions)

Sample query for nested objects and geo_spatial:

{ "where": { "and": [ { "address.geo": { "geo_shape": { "address.geo": { "shape": { "type": "circle", "coordinates": [ 78.38214860577659, 17.44064909837838 ], "radius": "1km" }, "relation": "within" } } } }, { "entity_properties.value": { "gt": 50 } } ] } }

entity_properties is a nested object and address is an object.
address.geo has geo_shape data.

Bharath Reddy Kontham

@bharathkontham

Updated example for Elasticsearch 6.x in https://github.com/bharathkontham/loopback-connector-elastic-search

Please check examples/server/datasources.sample-es-6.js

Pulkit Singhal

@pulkitsinghal

@bharathkontham - would you like to be added as a contributor to the strongloop community repo so that you can own and release 6.x related work more quickly?

Since you will only focus on 6.x and I cannot just nuke the other versions ... We can create a branch for 6.x development and perform 6.x releases from it or I am open to other ideas too.

Let us know.

Bharath Reddy Kontham

@bharathkontham

@pulkitsinghal - I would like to work on that, add me to strongloop community.

Pulkit Singhal

@pulkitsinghal

Soon I will cut a branch
Ask for you to be added as a contributor
Document which branch to merge your PRs into for your 6.x work ... also for any other users and contributors that come along.

Can’t commit to a imprint but that’s the plan :)

Jeff Hoffer

@eudaimos

@pulkitsinghal can you please help me understand the version tagging in this repo? Why are there v1.3.x tags and 1.3.x tags with the same version, both of which will show a commit making the same change in the package.json version field to the same value?

Shubham Thakur

@sbmthakur

@pulkitsinghal Hi Pulkit, you had removed the index key from datasource-test-v6-plain.json(sbmthakur-master branch). After restoring it all filter tests passed. I am unable to comprehend your intention behind that change. Was it done to test any breaking change in ES 6?

Pulkit Singhal

@pulkitsinghal

@eudaimos publishing might have failed midway with npm-release so the second attempt may have occurred with npm publish with a minor difference in the syntax where v was omitted or vice versa

@sbmthakur in ESv6, types are no longer supported ... what used to be a type in the past should either get its own index or it should be implemented by the connector as a field inside the document ... either way, a global index is not universally applicable and it would be presumptuous of me to continue with that as the default approach ... with such thoughts in mind I removed it and tried to play around with tests ... the failures are meant to show where more work is needed in the absence of a universal index for all types

Pulkit Singhal

@pulkitsinghal

https://www.elastic.co/guide/en/elasticsearch/reference/6.x/breaking-changes-6.0.html

Pulkit Singhal

@pulkitsinghal

@bharathkontham - we are hosting a meetup on Elasticsearch and loopback in Pune India. Tentatively it’s planned for sometime between March 2nd and March 14th.

I would like to invite you to attend as a speaker and present and talk about whatever you want on these topics.

The Airfair to fly from Hyderabad to pune and back will be at no cost to you.

Cc @aquid

Jeff Hoffer

@eudaimos

thanks @pulkitsinghal but it's consistent across each 1.3.* release - also I see that there's a commit with a message of release/1.4.1 but there is not tag for 1.4.1 - was this intentional?

Pulkit Singhal

@pulkitsinghal

I don’t remember so I’ll add it to my todo list to dig into this and get back to you ... in the meantime let me know how this is blocking you so I gain a better sense of priority

Bharath Reddy Kontham

@bharathkontham

@pulkitsinghal - I'll attend the meetup in pune.
Can you create a branch for 6.x work with es-6.x or any name

Pulkit Singhal

@pulkitsinghal

@bharathkontham - treat feature/esv6 as your "develop" ... create feature/esv6/featurename branches from it and PR/merge them back to feature/esv6 and the releases branches can be used to publish the package under the name loopback-connector-esv6 on npmjs

Bharath Reddy Kontham

@bharathkontham

loopback-connector-esv6 updated to 1.0.1.
check release notes here:
https://github.com/strongloop-community/loopback-connector-elastic-search/releases/tag/1.0.1

Bharath Reddy Kontham

@bharathkontham

Released v 1.0.3 for https://www.npmjs.com/package/loopback-connector-esv6

Fixed "limit" and "include" filter issues when used without "where" filter

Pulkit Singhal

@pulkitsinghal

Nice

Anouar Kacem

@anouarkacem

Hi guys I have an issue with auth

        "protocol": "https",
        "host": "scalr.api.appbase.io",
        "port": "443",
        "auth": "username:password"
      }

im trying to connect to appbase which is an elastic search server however in my debugger i have this result

Elasticsearch DEBUG: 2018-03-28T06:17:17Z
starting request {
"method": "HEAD",
"castExists": true,
"path": "/catalogdev",
"body": {},
"query": {}
}
Elasticsearch TRACE: 2018-03-28T06:17:19Z
-> HEAD https://scalr.api.appbase.io:443/catalogdev
{}
<- 400
Elasticsearch DEBUG: 2018-03-28T06:17:19Z
Request complete
Connection fails: Bad Request :: {"path":"/catalogdev","query":{},"body":"{}","statusCode":400,"response":""}

using curl https://username:password@scalr.api.appbase.io:443/catalogdev works fine

Anouar Kacem

@anouarkacem

any idea why its not authenticating ? thank you

Bharath Reddy Kontham

@bharathkontham

@anouarkacem It's not an authentication issue.
appbase.io is not supporting all elasticsearch APIs. In the debug info it is HEAD API which is not supported by appbase.io

Anouar Kacem

@anouarkacem

@bharathkontham thank you for your answer

Pulkit Singhal

@pulkitsinghal

👏

Siddharth Kothari

@siddharthlatest

Hey everyone, @bharathkontham reached out about the issue and we have just added support for the HEAD APIs. @anouarkacem can you verify if this works now?

Anouar Kacem

@anouarkacem

@siddharthlatest well I updated the package but the have same error, also I can't see the PR on github can you please share it here, thank you for reaching out

Bharath Reddy Kontham

@bharathkontham

@anouarkacem there is no PR for this. appbase.io added support for Index HEAD API.
can you share the connector version? is it esv6 or es connector?
https://rest.appbase.io/#2e9a1a76-ca3d-da8e-2cf0-5c5b793038c9

Anouar Kacem

@anouarkacem

@bharathkontham I got it, but I am not sure what it doesn't work I have the same result

Siddharth Kothari

@siddharthlatest

@anouarkacem sorry to hear. If you or someone here would be willing to create a sandbox code (or a repl) snippet, I can have someone from our team take a look if this is an issue from our end.

Samuel Bednarčík

@samuelbednarcik_twitter

hello, is there any way to build a nested query using this connector?

Bharath Reddy Kontham

@bharathkontham

@samuelbednarcik_twitter yes, you can.
sample:

{ "where": { "and": [ { "address.geo": { "geo_shape": { "address.geo": { "shape": { "type": "circle", "coordinates": [ 78.38214860577659, 17.44064909837838 ], "radius": "1km" }, "relation": "within" } } } }, { "entity_properties.value": { "gt": 50 } } ] } }

address and entity_properties in above query are nested objects

Samuel Bednarčík

@samuelbednarcik_twitter

it does not work, buildFilter debug output is

{"index":"user_service_account","type":"_doc","body":{"sort":["_uid"],"query":{"bool":{"must":[{"range":{"entity_properties.value":{"gt":50}}}]}}},"size":10}

Samuel Bednarčík

@samuelbednarcik_twitter

a just find out that i was using wrong version of the package, not for elasticsearch 6

ghazaltaimur13

@ghazaltaimur13

hi guys, i need to send a data to two databases / data source. Is this possible? and also run migration in both the databases? My datasource type is mysql

Pulkit Singhal

@pulkitsinghal

@kamal0808 might have something to say on this topic

pookdeveloper

@pookdeveloper

@pookdeveloper
Hello how can i defined only user with role admin can create a new user???
i try this but not work..
{
"accessType": "EXECUTE",
"principalType": "ROLE",
"principalId": "admin",
"permission": "ALLOW",
"property": "create"
},
i am a nob in loopback .. sorry :(

Alex Pepper

@alexanderpepper

Hi all, I'm looking to understand what the index property represents in the datasources.json definition

  "elasticSearchDB": {
    "connector": "es",
    "name": "elasticSearchDB",
    "index": "whatGoesHere?", ...

My understanding is an index is something defined by a mapping, but this is outside of the mappings property

Alex Pepper

@alexanderpepper

^ Figured out my question

Bharath Reddy Kontham

@bharathkontham

Hi all,

Updated Elastcsearch 6.x connector to 1.1.0
https://www.npmjs.com/package/loopback-connector-esv6

@pulkitsinghal / administrators, can we move old module which supports 5.x and older to 5.x branch and merge esv6 to master to make it default.

I'm not seeing any active development for 5.x module

Bharath Reddy Kontham

@bharathkontham

Hi all,

published Elasticsearch 6.x connector 1.2.0
https://www.npmjs.com/package/loopback-connector-esv6

added support for replaceById and replaceOrCreate methods

pookdeveloper

@pookdeveloper

@pookdeveloper
Hello how can i defined only user with role admin can create a new user???
i try this but not work..
{
"accessType": "EXECUTE",
"principalType": "ROLE",
"principalId": "admin",
"permission": "ALLOW",
"property": "create"
},
i am a nob in loopback .. sorry :(
I have already do what i want

Hello, any way to use a File for all logs in loopback ??? for my production app ???

rhlr

@rhlr

Is there any way to combine near queries with multi_match ? I'm trying to use and operator but the multi_match query has no effect on the result.

{"where":{"and":[{"geo":{"near":{"lat":30.78,"lng":76.69}}},{"native":{"query":{"multi_match":{"query":"python","fields":["title","tags","description"],"type":"most_fields"}}}}]}}

Where communities thrive

People

Repo info

Activity