SoftInstigate/restheart

RESTHeart, Instant RESTFul API for MongoDB

Maurizio Turatti

@mkjsix

You need to check that Cosmos is detected as at least MongoDB 3.4 wire protocol version 5, which is a preview feature you have to explicitly activate on the Cosmos instance

1 reply

Amol Ingale

@aingale.amol_gitlab

13:07:29.342 [main] ?[34mINFO ?[0;39m o.r.s.p.a.MongoRealmAuthenticator - No user found. Created default user with _id BsonString{value='admin'}
13:07:31.327 [main] ?[34mINFO ?[0;39m o.r.mongodb.db.MongoClientSingleton - Connecting to MongoDB...
13:07:31.635 [main] ?[34mINFO ?[0;39m o.r.mongodb.db.MongoClientSingleton - MongoDB version ?[35m3.6.0?[m
13:07:31.635 [main] ?[31mWARN ?[0;39m o.r.mongodb.db.MongoClientSingleton - MongoDB is a standalone instance.
13:07:31.766 [main] ?[34mINFO ?[0;39m org.restheart.mongodb.MongoService - ?[32mURI / bound to MongoDB resource /restheart?[m
13:07:31.920 [main] ?[34mINFO ?[0;39m org.restheart.Bootstrapper - HTTP listener bound at 0.0.0.0:8080
13:07:31.921 [main] ?[39mDEBUG?[0;39m org.restheart.Bootstrapper - Content buffers maximun size is 16777216 bytes
13:07:31.934 [main] ?[34mINFO ?[0;39m org.restheart.Bootstrapper - ?[32mURI / bound to service mongo, secured: true?[m
13:07:31.935 [main] ?[34mINFO ?[0;39m org.restheart.Bootstrapper - ?[32mURI /ic bound to service cacheInvalidator, secured: false?[m
13:07:31.936 [main] ?[34mINFO ?[0;39m org.restheart.Bootstrapper - ?[32mURI /csv bound to service csvLoader, secured: true?[m
13:07:31.936 [main] ?[34mINFO ?[0;39m org.restheart.Bootstrapper - ?[32mURI /roles bound to service roles, secured: true?[m
13:07:31.936 [main] ?[34mINFO ?[0;39m org.restheart.Bootstrapper - ?[32mURI /ping bound to service ping, secured: false?[m
13:07:31.937 [main] ?[34mINFO ?[0;39m org.restheart.Bootstrapper - ?[32mURI /tokens bound to service rndTokenService, secured: false?[m
13:07:31.937 [main] ?[39mDEBUG?[0;39m org.restheart.Bootstrapper - No proxies specified
13:07:31.945 [main] ?[39mDEBUG?[0;39m org.restheart.Bootstrapper - Allow unescaped characters in URL: true

Andrea Di Cesare

@ujibang

As far as I remember with cosmos dB you need to create the db via the azure console and mount it on RESTHeart configuration file (by default it binds / with the db "restheart")

kbowens

@kbowens

@ujibang Thanks so much!

Gabi Hassan

@gabihassan

hello

i have a doubt

how to create a route? example?

now:
localhost:8080/data?filter{email: /^email.*/}

to:
localhost:8080/data?email=a

Andrea Di Cesare

@ujibang

Hi @gabihassan , you can use a request interceptor to modify the request creating the filter from the query parameter

Ben S

@benjsec

Hi all, I'm having some difficulty getting restheart to connect to Amazon's DocumentDB. I've been following the guides at https://restheart.org/docs/security/secure-connection-to-mongodb/ and https://medium.com/softinstigate-team/how-to-create-a-web-api-for-aws-documentdb-using-restheart-987921df3ced but keep getting an HTTP 500 (Timeout connecting to MongoDB, is it running?). When starting up the logs also mention not being able to get the mongodb version which I suspect is related.
I can connect using other tools (mongo shell and pymongo) without any issue, using the same URI and RDS certificate bundle. Is there anything else that I can check to see what's going wrong? The logs are frustratingly quiet on the mongo side of things despite having set the loglevel to ALL.
If it's relevant this is using an older version (3.0.0) due to downstream dependencies.

Andrea Di Cesare

@ujibang

Hi Ben, Timeout connecting to MongoDB means RESTHeart is not able to reach DocumentDB. Double check the mongo-uri.

Did you open port 27017 in the VPC security group?

The last step is to modify the VPC’s Security Group, in order to open the port 27017, to give database access to RESTHeart.

Andrea Di Cesare

@ujibang

Did you import rds-combined-ca-bundle.pem into the java keystore?

is your mongo-uri in restheart conf something like (note ssl=true): mongodb://restheart:restheart00@docdb-restheart-2019–05–06.cluster-ccexczqns5xm.eu-west-1.docdb.amazonaws.com:27017/ssl=true

Ben S

@benjsec

Hi, sorry - gitter tab had closed in the background. I've double and tripple checked the URI and used the same URI with both mongo-shell and pymongo so am confident the connectivity is there, and it does have the ssl=true section.
For the .pem file I tried importing the whole thing into a keystore as in your docs, and also splitting it into individual certificates to import as in the AWS documentation.

However I've also just found that documentDB doesn't seem to support map reduce, which means we're going to have to revert to mongo anyway.

Andrea Di Cesare

@ujibang

Ah! Yes document dB doesn't have the full features of MongoDb. If you plan using Atlas, we are partner of MongoDb and can refer you to MongoDb sales. Let me know in case...

Andrea Di Cesare

@ujibang

Hi @benjsec we found an issue on the tutorial. Looking at this stackoverflow question https://stackoverflow.com/questions/59739006/aws-documentdb-tls-connection-with-java

we got that we..

have had to stop using the bundled option (rds-combined-ca-bundle.pem) and start using this one rds-ca-2019-root.pem https://s3.amazonaws.com/rds-downloads/rds-ca-2019-root.pem

we will be updating the tutorial...

akernan

@akernan

Noob Question: is there a tutorial / example somewhere that further explains Change Streams? I am essentially looking to collate a local "master" database from remote sensor data. On the surface, ChangeStream looks like this could update the Master regularly... am i off base?

Master.db < ===== [ Remote.db <---Sensor Data ]

Andrea Di Cesare

@ujibang

Hi @akernan , change streams allow to send real time data to clients over WebSocket protocol. So your use case seems feasible but you need a client that receives data from the remote db and writes it to the master.

An easier alternative that does not require another component in your architecture is implementing an hook developing an Interceptor.

Documentation is at https://restheart.org/docs/plugins/core-plugins/#interceptors

Basically with the following code

@RegisterPlugin(name = "hook", 
    interceptPoint = InterceptPoint.RESPONSE_ASYNC,
    description = "an hook ...")
public class ReadOnlyPropFilter implements MongoInterceptor {
    @Override
    public void handle(MongoRequest request, MongoResponse response)  {

     ….
    }

If the sensor wirtes data using RESTHeart, the interceptor's handle() method would be invoked on writes and you can easily send a POST to the master db to replicate data...

Edgar Silva

@skalena_gitlab

Hi there, any chance to chance the default context in Docker?

Edgar Silva

@skalena_gitlab

Such as, instead of to be / to be something like /restheart

Edgar Silva

@skalena_gitlab

I did solved

Andrea Di Cesare

@ujibang

Hi @skalena_gitlab i was going to point you to restheart.yml conf file, following section

mongo-mounts:
  - what: {{{root-mongo-resource}}}
    where: /restheart <----

just set mongo-mounts/where to /restheart

Edgar Silva

@skalena_gitlab

Thanks @ujibang

Koteswara Rao Molabanti

@koteswararao1_gitlab

Hi, need help how to use Aggregations in restheart.
below is aggregations. please provide suggestion
db.getCollection("graphData").aggregate(
[
{
"$match" : {
"timestamp" : {
"$gte" : ISODate("2020-09-01T05:20:46.111+0000"),
"$lt" : ISODate("2020-09-09T05:20:46.111+0000")
},
"badger_id" : "0x90227008",
"sensor_id" : "3"
}
},
{
"$group" : {
"_id" : {
"$add" : [
{
"$subtract" : [
{
"$subtract" : [
{
"$add" : [
"$timestamp",
36000000.0
]
},
ISODate("1970-01-01T00:00:00.000+0000")
]
},
{
"$mod" : [
{
"$subtract" : [
{
"$add" : [
"$timestamp",
36000000.0
]
},
ISODate("1970-01-01T00:00:00.000+0000")
]
},
7200000.0
]
}
]
},
ISODate("1970-01-01T00:00:00.000+0000")
]
},
"sensor_id" : {
"$first" : "$sensor_id"
},
"seq" : {
"$first" : "$seq"
},
"timestamp" : {
"$first" : "$timestamp"
},
"temp_f" : {
"$first" : "$temp_f"
},
"temp_c" : {
"$first" : "$temp_c"
},
"badger_id" : {
"$first" : "$badger_id"
},
"sensorgroup_id" : {
"$first" : "$sensorgroup_id"
},
"moisture" : {
"$first" : "$moisture"
}
}
}
],
{
"allowDiskUse" : false
}
);

Andrea Di Cesare

@ujibang

Hi @koteswararao1_gitlab, please refer to the documentation section about aggregations https://restheart.org/docs/aggregations/

in short, you define the aggregation as a metadata property of the collection.

an example follows:

PUT /coll HTTP/1.1

{ "aggrs" : [ 
      { "stages" : [ { "$match" : { "name" : { "$var" : "n" } } },
            { "$group" : { "_id" : "$name",
                  "avg_age" : { "$avg" : "$age" }
                } }
          ],
        "type" : "pipeline",
        "uri" : "example-pipeline"
      },
      { "map" : "function() { emit(this.name, this.age) }",
        "query" : { "name" : { "$var" : "n" } },
        "reduce" : "function(key, values) { return Array.avg(values) }",
        "type" : "mapReduce",
        "uri" : "example-mapreduce"
      }
    ] }

This way the result of the aggregation is returned with GET /coll/_aggrs/example-pipeline

Koteswara Rao Molabanti

@koteswararao1_gitlab

Hi Andrea, so the aggregation json should be passed in body of PUT request right ?

Andrea Di Cesare

@ujibang

yes PUT /coll defines metadata of the collection coll (that you can read with GET /coll/_meta

in metadata of collection, you can define aggregations

Koteswara Rao Molabanti

@koteswararao1_gitlab

thanks Andrea.

Damien

@dev-indb

Hi.
I am currently migrating my solution from restheart 3.10.1 to version 5. I wrote a Custom Access Manager https://restheart.org/docs/v3/custom-access-manager/
Do I need to transform my component into a plug-in?

Andrea Di Cesare

@ujibang

yes! implement it as a plugin and just deploy it copying the jar to the /plugins directory

Andrea Di Cesare

@ujibang

Access Manager is now called Authorizer https://restheart.org/docs/plugins/security-plugins/#authorizers

the interface are very similar, so your migration should be straightforward. Have a look at repo https://github.com/softInstigate/restheart-examples for few plugins examples. Althrough you won't find an example access manager, you'll find detailed instructions on how to build and deploy a plugin...

@dev-indb , if you need futher help, just ask!

Damien

@dev-indb

Thanks Andrea.

Damien

@dev-indb

Where do I find the equivalent of RequestContext ?
public boolean isAllowed(HttpServerExchange exchange, RequestContext context)
I'm trying to access to context.getDBName(), context.getCollectionName() and context.getDocumentId() getters.

Andrea Di Cesare

@ujibang

(good question, we'll add some documentation about that)

you see that now RequestContext has been replaced by Request and Response classes. In your case you have

boolean isAllowed(final Request request)

now the Request class has several subclasses. In case of a request handled by MongoService (the mongodb API) it is MongoRequest

if (request instanceof MongoRequest) {
  var mrequest = (MongoRequest) request;
  var dbName = mrequest.getDBName();
  var collName = mrequest.getCollectionName();
...
}

you can also get the HttpServerExchange with request.getExchange()

Damien

@dev-indb

Done !
Thanks for your help Andrea

Damien

@dev-indb

3 hours of work. Great !

Andrea Di Cesare

@ujibang

Super!

Where communities thrive

People

Repo info

Activity