Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 22 15:56
    albogdano closed #280
  • Jan 22 15:56

    albogdano on master

    added delete protection for val… (compare)

  • Jan 22 14:19

    albogdano on master

    added support for uploading ava… (compare)

  • Jan 22 14:18

    albogdano on master

    added support for uploading ava… (compare)

  • Jan 21 17:32

    albogdano on master

    minor fixes around avatar uploa… (compare)

  • Jan 19 16:41

    albogdano on master

    added ImgurAvatarRepository, re… (compare)

  • Jan 19 13:37

    albogdano on master

    fixed space selection menu not … (compare)

  • Jan 18 21:11

    albogdano on master

    changed order of all spaces in … (compare)

  • Jan 18 21:06

    albogdano on master

    fixed all spaces label (compare)

  • Jan 18 20:55

    albogdano on master

    changed all spaces to be the de… (compare)

  • Jan 18 20:26
    albogdano commented #279
  • Jan 18 20:22

    albogdano on master

    fixed avatar-related tests (compare)

  • Jan 18 20:15
    albogdano commented #278
  • Jan 18 20:13
    albogdano closed #278
  • Jan 18 20:13

    albogdano on master

    Add enum for each size already … Move macro profilepic to Scoold… Fix endline and 26 more (compare)

  • Jan 18 20:12
    albogdano synchronize #278
  • Jan 18 06:50
  • Jan 18 06:50
    okanchou9 starred Erudika/scoold
  • Jan 17 16:25
    Orgise starred Erudika/scoold
  • Jan 16 08:17
    vasvl123 starred Erudika/scoold
pikrakpzu
@pikrakpzu
Perfect, had to edit multiple times to get table rendered right :smile:
Alex Bogdanovski
@albogdano
@pikrakpzu ok, let me know what you find. And, yes MongoDB is used by Para to store data so you are in fact using it.
I am not quite sure how we can fix your problematic users in this situation but definitely check the user objects in Para and if they have the correct user data from OIDC
pikrakpzu
@pikrakpzu

@albogdano At last I had some time to investigate Scoold users.
Looks like there is a problem with profile update.
While user was updated with correct name, profile have old wrong name and new correct originalName.

Method I used to check this.

Search query:
para-cli search "creatorid:ID* OR id:ID*"
Where ID is a id field value from result of:
para-cli search "type:user AND email:greatuser*"

For users that used ldap and oa2 to login, result is (excluding posts/queries/votes etc.):

sysprop | identifier | id:ldap:...
sysprop | identifier | id:oa2:...
type | profile
type | user

On profile we have name and originalName.
profile where identifier for ldap was created before oa2 got valid name and originalName seems ok.
profile where identifier for oa2 was created before ldap (and before last update which fixed oa2) got invalid name but originalName seems ok.

Also I have found that long time ago I had different name pattern from ldap like User Name (login) and oa2 is User Name (Office).
My own profile is one of oldest and it has name in old format, but originalName have been updated from oa2.
Last time I used oa2 to signin.

Seems that Scoold uses name field from profile which is not updated with data from identity providers (ldap, oa2).
I didn't write about user object as it contains valid name.
If it would help I can paste this objects.

Alex Bogdanovski
@albogdano
@pikrakpzu yes, there's an option for that - para.name_edits_enabled = false this way, you disable name editing on Scoold and force name updates to be passed on from the identity provider (LDAP or OA2)
the idea is that once a Scoold account is created (type: profile) users can edit their name independently from what's provided by the IDP. For better synchronization, you can turn off that feature and always sync the names
I'm sorry if I forgot to mention that earlier in our conversation... :neutral_face:
could've saved you some time
pikrakpzu
@pikrakpzu
@albogdano That's exactly what I need, i'm glad we finally get to it :smile:
rafik777
@rafik777
@albogdano How to extend the default session duration? Token expiration? I am using LDAP login. I have to sing in too often...
Alex Bogdanovski
@albogdano
@rafik777 for example, set para.session_timeout = 21600 for 6h session duration (in seconds) and restart. In 6h the auth cookie will
expire but the JWT inside the cookie may still be valid. So make sure you also configure Para to issue JWTs with the same
validity period using para.jwt_expires_after = 21600 or by updating your Para app object to have a property tokenValiditySec: tokenValiditySec
se-alexnsa
@se-alexnsa
Hey there. Whenever we restart the para service (in AWS Fargate, using RDS mysql to store database), we have a the server is unhealthy and must be re-indexed/re-built. Each time, we use para-cli rebuild-index and it works - it gets re-indexed and the server is healthy again.
To save us from running that para-cli command every time, will changing the para config to use the para elasticsearch plugin work to prevent that re-indexing error coming up? and will we have to also spin up an AWS Elasticsearch service to use alongside the plugin?
Alex Bogdanovski
@albogdano
@se-alexnsa that is not normal - there must be something wrong with the Par configuration or the search index storage. Are you sure the server is unhealthy because the index has been lost? What's the exact message in the logs?
If you use the ES plugin, you'll definitely need to connect to some Elasticsearch server of your own
se-alexnsa
@se-alexnsa

Yes, will set up an Elasticsearch server

Regarding the server unhealthy message, it was this

2021-11-02 10:56:11 [WARN ] Server is unhealthy - the search index may be corrupted and may have to be rebuilt.

then after I ran para-cli rebuild-index, the log message was

[INFO ] Server is healthy.

Alex Bogdanovski
@albogdano
@se-alexnsa I'm pretty sure that your search index is not stored to disk properly. check if the ./data folder is mounted properly as a volume if using docker.
Alex Bogdanovski
@albogdano
@pikrakpzu in the next release of Scoold, you'll be able to make the feedback page available only to registered users by setting para.is_default_space_public = false
se-alexnsa
@se-alexnsa

@se-alexnsa I'm pretty sure that your search index is not stored to disk properly. check if the ./data folder is mounted properly as a volume if using docker.

We're using AWS Fargate.

When running locally using docker, data is at /para/data
I didn't physically mount it, but it was already there when I spun up the container. Do we need to do something differently there?
And how can that be mounted when using Fargate too?

Alex Bogdanovski
@albogdano
@se-alexnsa I am no expert in Fargate but there should be a volumes section in the settings. my guess is that Fargate doesn't allow local disk writes and all writable volumes must be attached explicitly.
se-alexnsa
@se-alexnsa
OK. Also, if I install the Elasticsearch plugin as per the docs https://github.com/erudika/para-search-elasticsearch, do we also have to add the erudikaltd/para-search-elasticsearch image to our already existing dockerfile?
Alex Bogdanovski
@albogdano
@se-alexnsa yes, you need to add the plugin image as a layer on top of the Para Docker image. Also pay attention to where the ./data volume is mounted - without it the indexing won't work
se-alexnsa
@se-alexnsa
locally the /data volume is mounted in /para directory
/para/data. That was without me having to manually mount it.
Should /data be mounted in /para for the Fargate container too?
Alex Bogdanovski
@albogdano
@se-alexnsa yes, that is the default location and it should work on Fargate as well
pikrakpzu
@pikrakpzu

@pikrakpzu in the next release of Scoold, you'll be able to make the feedback page available only to registered users by setting para.is_default_space_public = false

Thanks, I've just done tests and going to prod with 1.46.5 :smile:

pikrakpzu
@pikrakpzu

@albogdano Hello! :)
My organization's security team found few bugs in Scoold.

First is with avatar url set by users.

There's no check of what user set, and its served to all other users as avatar.
User can setup malicious link, containing js code which for exapmle could steal auth tokens from cookies.
As example this URL were given:

https://suvroc.github.io/security-demos/XSS/reflectedXSS.html?name=%3Cscript%3Ealert%3C/script%3E

Recommendations:

  • User input data filtering, for example content-type check.
  • Implementing CORS(Cross-Origin Resource Sharing)
  • Secure headers
  • Implementing CSP(Content Security Policy)

References:

Second is about session.

Cookie scoold-auth with JWT can be stolen and it's valid for 7 days.
Also user session is not closed after logout, with stolen cookie it's possible to get access without login.
Recommendations:

References:

I've seen para.jwt_expires_after few posts before.
Correct me if i'm wrong, when set to 300 it will make JWT expire after 5min? It should fix 7 days problem.
But it won't solve existing session after logout.

Both problems were marked as high security risk.

Is there anything that can be done to fix them?

Alex Bogdanovski
@albogdano

@pikrakpzu Hey, thanks for this! I will work on the code to address these points.

  • On the first topic: I don't think that's even possible because Scoold has a pretty strict CSP - https://cspvalidator.org/#url=https://live.scoold.com
    Security headers are also in place - https://securityheaders.com/?q=https%3A%2F%2Flive.scoold.com&followRedirects=on
    There's always room for improvement here and I will soon add filtering of the actual URLs for avatars

  • On the second topic: Scoold does not use sessions at all - it uses JWTs instead. It's a similar concept and JWTs can be made to expire after a configurable time period.
    para.session_timeout is the validity period in seconds for the auth cookie itself, para.jwt_expires_after is the validity period in seconds for the JWT token inside the auth cookie.
    Again, here we can tighten security by only allowing one valid JWT per user/browser. It should be pretty straightforward to implement this.

se-alexnsa
@se-alexnsa

yes, you need to add the plugin image as a layer on top of the Para Docker image

If I do that, do I still need to download this jar file as well:
https://github.com/Erudika/para-search-elasticsearch/releases

Alex Bogdanovski
@albogdano
@se-alexnsa no need - the dockerfile for the ES plugin will download the JAR into the lib folder next to para.jar
se-alexnsa
@se-alexnsa

great
also since mounting the volume, we no longer get indexing errors, but now instead have this error

Application run failed org.springframework.context.ApplicationContextException: Failed to start bean 'webServerStartStop'; nested exception is org.springframework.boot.web.server.WebServerException:

could this be because something has changed location now?

Perhaps something else needs to be mounted manually now as well?
Alex Bogdanovski
@albogdano
@se-alexnsa I will need to see the full stack trace to tell you what caused that exception.
se-alexnsa
@se-alexnsa
      ____  ___ _ ____ ___ _ 
     / __ \/ __` / ___/ __` /
    / /_/ / /_/ / /  / /_/ / 
   / .___/\__,_/_/   \__,_/  v1.40.1-SNAPSHOT
  /_/                        

2021-11-24 14:08:45 [INFO ] --- Para.initialize() [production] ---
2021-11-24 14:08:45 [INFO ] Loaded new DAO, Search and Cache implementations - SqlDAO, LuceneSearch and CaffeineCache.
2021-11-24 14:08:46 [INFO ] HikariPool-1 - Starting...
2021-11-24 14:08:46 [INFO ] HikariPool-1 - Start completed.
2021-11-24 14:08:51 [INFO ] Server is healthy.
2021-11-24 14:08:51 [INFO ] Found root app 'para' and 0 existing child app(s).
2021-11-24 14:08:55 [INFO ] Queue 'para-default' could not be found: software.amazon.awssdk.services.sqs.model.QueueDoesNotExistException: The specified queue does not exist for this wsdl version. (Service: Sqs, Status Code: 400, Request ID: e7c38864-0824-594a-91fd-d27c419227a6, Extended Request ID: null)
2021-11-24 14:08:55 [ERROR] null
java.util.concurrent.ExecutionException: software.amazon.awssdk.services.sqs.model.SqsException: Access to the resource https://sqs.eu-west-1.amazonaws.com/ is denied. (Service: Sqs, Status Code: 403, Request ID: 08a4be23-07df-519e-9d7e-e43b4e68cf72, Extended Request ID: null)
    at java.base/java.util.concurrent.CompletableFuture.reportGet(Unknown Source)
    at java.base/java.util.concurrent.CompletableFuture.get(Unknown Source)
    at com.erudika.para.queue.AWSQueueUtils.createQueue(AWSQueueUtils.java:95)
    at com.erudika.para.queue.AWSQueue.getUrl(AWSQueue.java:89)
    at com.erudika.para.queue.AWSQueue.startPolling(AWSQueue.java:65)
    at com.erudika.para.ParaServer.initialize(ParaServer.java:157)
    at com.erudika.para.ParaServer.runAsJAR(ParaServer.java:423)
    at com.erudika.para.ParaServer.main(ParaServer.java:432)
    at com.erudika.para.Run.main(Run.java:26)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.base/java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
    at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:467)
Caused by: software.amazon.awssdk.services.sqs.model.SqsException: Access to the resource https://sqs.eu-west-1.amazonaws.com/ is denied. (Service: Sqs, Status Code: 403, Request ID: 08a4be23-07df-519e-9d7e-e43b4e68cf72, Extended Request ID: null)
    at software.amazon.awssdk.services.sqs.model.SqsException$BuilderImpl.build(SqsException.java:95)
    at software.amazon.awssdk.services.sqs.model.SqsException$BuilderImpl.build(SqsException.java:55)
    at software.amazon.awssdk.protocols.query.internal.unmarshall.AwsXmlErrorUnmarshaller.unmarshall(AwsXmlErrorUnmarshaller.java:99)
    at software.amazon.awssdk.protocols.query.unmarshall.AwsXmlErrorProtocolUnmarshaller.handle(AwsXmlErrorProtocolUnmarshaller.java:102)
    at software.amazon.awssdk.protocols.query.unmarshall.AwsXmlErrorProtocolUnmarshaller.handle(AwsXmlErrorProtocolUnmarshaller.java:82)
    at software.amazon.awssdk.core.http.MetricCollectingHttpResponseHandler.lambda$handle$0(MetricCollectingHttpResponseHandler.java:52)
    at software.amazon.awssdk.core.internal.util.MetricUtils.measureDurationUnsafe(MetricUtils.java:64)
    at software.amazon.awssdk.core.http.MetricCollectingHttpResponseHandler.handle(MetricCollectingHttpResponseHandler.java:52)
    at software.amazon.awssdk.core.internal.http.async.AsyncResponseHandler.lambda$prepare$0(AsyncResponseHandler.java:89)
    at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(Unknown Source)
    at java.base/java.util.concurrent.CompletableFuture.postComplete(Unknown Source)
    at java.base/java.util.concurrent.CompletableFuture.complete(Unknown Source)
    at software.amazon.awssdk.core.internal.http.async.AsyncResponseHandler$BaosSubscriber.onC
at the top, it says a sqs queue is missing (para-default?), but we've never needed to make a queue before in order to get it working
Alex Bogdanovski
@albogdano
@se-alexnsa do you set para.webhooks_enabled = true or para.q = "sqs" anywhere in your configuration?
se-alexnsa
@se-alexnsa
this only is appearing after mounting the volume, so I wondered if I have to now manually mount other things as well
I'll check...
Yes, para.webhooks_enabled = true is set in para application.conf
Alex Bogdanovski
@albogdano
ok, please set para.q = "LocalQueue"
then restart Para
se-alexnsa
@se-alexnsa
ok
and this will still work ok in AWS Fargate?
Alex Bogdanovski
@albogdano
yes, most certainly
se-alexnsa
@se-alexnsa
should i also remove para.webhooks_enabled = true , or set it to false instead?
Alex Bogdanovski
@albogdano
you can keep the webhooks enabled if you are using them but otherwise I would advise you to disable them for extra security
se-alexnsa
@se-alexnsa
I have tried it i.e. set para.q = "LocalQueue" and restarted para, but the same error about a missing queue remains
Alex Bogdanovski
@albogdano
@se-alexnsa hm, strange - how about para.q = "local"?
you may want to update Para to 1.41.3
se-alexnsa
@se-alexnsa
ok great it works now thanks
Alex Bogdanovski
@albogdano
@se-alexnsa great! I'm glad I could help.
se-alexnsa
@se-alexnsa
What do we need to do to resolve this error?
Invalid JWT found in cookie scoold-auth
Alex Bogdanovski
@albogdano
@se-alexnsa Are you using ParaIO.com?
se-alexnsa
@se-alexnsa
no