Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Sep 30 13:27

    albogdano on snyk-upgrade-d5c9be2ee66b5ac7a3cebefe51909cb2

    (compare)

  • Sep 30 13:27
    albogdano closed #166
  • Sep 30 13:27

    albogdano on snyk-upgrade-ba2eeac3d61fb17ca7bcfe0078e67072

    (compare)

  • Sep 30 13:27
    albogdano closed #168
  • Sep 30 13:27

    albogdano on snyk-upgrade-ad66dca741fb1a4d122ee777b5849c14

    (compare)

  • Sep 30 13:27
    albogdano closed #164
  • Sep 30 13:26

    albogdano on snyk-upgrade-33d704c82a31a38ea9d3223290aeea9e

    (compare)

  • Sep 30 13:26

    albogdano on master

    updated Spring Boot to 2.7.4 (compare)

  • Sep 30 13:26
    albogdano closed #165
  • Sep 29 05:57
    snyk-bot opened #168
  • Sep 29 05:57

    albogdano on snyk-upgrade-ba2eeac3d61fb17ca7bcfe0078e67072

    fix: upgrade ch.qos.logback:log… (compare)

  • Sep 29 05:57

    albogdano on snyk-upgrade-ba2eeac3d61fb17ca7bcfe0078e67072

    (compare)

  • Sep 28 20:27
    setop starred Erudika/para
  • Sep 28 10:48

    albogdano on snyk-upgrade-ba2eeac3d61fb17ca7bcfe0078e67072

    (compare)

  • Sep 28 10:48
    albogdano closed #167
  • Sep 28 05:46
    snyk-bot opened #167
  • Sep 28 05:46

    albogdano on snyk-upgrade-ba2eeac3d61fb17ca7bcfe0078e67072

    fix: upgrade ch.qos.logback:log… (compare)

  • Sep 28 05:46

    albogdano on snyk-upgrade-ba2eeac3d61fb17ca7bcfe0078e67072

    (compare)

  • Sep 27 23:51
    albogdano opened #166
  • Sep 27 23:51

    albogdano on snyk-upgrade-d5c9be2ee66b5ac7a3cebefe51909cb2

    (compare)

Alex Bogdanovski
@albogdano
Both applications are very similar and based on Spring Boot. Perhaps you could just copy the Helm chart and update the container image to load erudikaltd/para:latest_stable instead of Scoold
Rob Bell
@admin82_gitlab
I was just thinking the same thing. I don't know much about Para and what other config and connection setup would be required if I were to roll them into a single chart, but I'll have a go at running Para on its own first
Rob Bell
@admin82_gitlab
Any pointers on how to get connection details for the para instance once it's deployed?
Alex Bogdanovski
@albogdano
@admin82_gitlab Para is deployed on port 8080 by default. the hostname is localhost.
I you use the histed service, the hostname is https://paraio.com
Rob Bell
@admin82_gitlab
I'm using an instance deployed to my cluster, but how do I get the key and secret once it's up and running? Do I need to manually create a database too?
Alex Bogdanovski
@albogdano
Rob Bell
@admin82_gitlab
So my pod should be provisioned with a Para container with a /para/data folder and an empty application.conf in /para?
then I'll need to exec into the pod to grab the written application.conf
Rob Bell
@admin82_gitlab
is there a method that would allow me to spin up both pods and have Scoold configured with the keys/secrets to communicate with Para?
Alex Bogdanovski
@albogdano
@admin82_gitlab well this is already possible with docker compose but I don't know how it can be achieved in K8
A completely automatic Helm chart for both Para and Scoold would be nice - i.e. one where both Para and Scoold get autoconfigured on first boot
But this is still something I haven't done yet
Rob Bell
@admin82_gitlab
Thanks—the docker-compose.yml is a useful starting point. If I can figure it out I'll make sure to share it
Alex Bogdanovski
@albogdano
:thumbsup:
Rob Bell
@admin82_gitlab

I'm having a little trouble passing configuration to the Para instance rather than letting it set its own configuration after setup—judging by the docs this sounds possible. I'd also like to avoid any imperative calls to create Para apps and configure these via passed config if possible. I'm not sure if either are achievable, but essentially the order of events in my Helm chart would be:

1) Define a k8s Secret object containing the Scoold app name, and a chosen access key and secret key
2) Pass the Secret to both the the Scoold and Para deployment/pods and have Para set this as the access key and secret rather than have it generate its own
3) Optionally run a pre-init container to create the Scoold app on the Para instance
4) Run both pods with the passed in Secret config for the access key/secret and app name

Is this possible or am I missing a simpler approach? The documentation for Para is a little sparse.

Alex Bogdanovski
@albogdano
@admin82_gitlab all secret keys for Para apps are auto-generated by Para. 1, 2, 4 are not possible. Para and Scoold have recently implemented auto-init functionality where each service will auto-initialize itself if the configuration file is missing. I'm not sure how config files are handled in K8 but passing preconfigured secrets is definitely not possible. You can always edit the database object and change the secret key but that can only be done manually. You can start Para, create the Scoold app and then pass that secret to the Scoold pod. In all cases, you will need to create a separate app for Scoold.
Rob Bell
@admin82_gitlab
The auto-init functionality should only refresh the application.conf and write a new root_access_key and root_secret_key if they're missing—is that correct? That's not what I've observed when spinning up a Para instance in Docker and mounting a volume with a populated application.conf
Config and Secret objects in K8s are readonly, so can be passed into pods but not re-written. The approach in K8s might be to provision Para as a StatefulSet as opposed to a Service, provide it a readwritemany PersistentVolumeClaim which the StatefulSet and the Scoold Deployment can mount and allow for writing and reading of credentials. I'm not sure this is a very safe method for sharing those though.
Rob Bell
@admin82_gitlab
Is it possible to use Azure AD for the auth from Scoold to Para, or is that only possible for authenticating users directly against Para?
Alex Bogdanovski
@albogdano
@admin82_gitlab ignore para.root_access_key and para.root_secret_key for now - they are just saved there for convenience when you need to use them. The secret key is never read by Para from the config file. Para generates a new secret each time an app is created and that secret is saved to the database.
The steps to initialize Scoold are:
  1. Initialize Para - create root app and secret
  2. Use the root app's secret to create new app for Scoold (either via API or Para CLI)
  3. Start Scoold with the keys generated in step 2
Azure AD/OIDC is supported and can be used as an identity provider.
Rob Bell
@admin82_gitlab
Thanks for the reply - I don't think the initialisation steps above will lend themselves well to a single Helm chart. Regarding Azure AD, can I use it both to allow users to log in to the Scoold app, and the Scoold app (via a Managed Identity or similar)
Alex Bogdanovski
@albogdano
@admin82_gitlab Azure AD can be used as the only identity provider in Scoold and you can disable password auth altogether. All profile data will be pulled from Azure then.
Rob Bell
@admin82_gitlab
And Scoold would authenticate with Para as the given Azure AD user, or there needs to be some other configuration of communication between Scoold and Para? Apologies, I'm a bit confused here
Alex Bogdanovski
@albogdano
@admin82_gitlab Scoold sends users to Azure for authentication, then Azure returns the request to Para where it is handled and finally Para redirects back to Scoold. Each Para app can have its own individual OAuth2 settings. Para handles all authentication requests.
truthmed1a
@truthmed1a
your stuff seems amazing!
Alex Bogdanovski
@albogdano
@truthmed1a Hey, thank you!
Lakshman
@lsornana
Hi. I'm trying configure Para to connect to Amazon DynamoDB instead of the DynamoDB on the localhost. In the Para documentation it says that setting para.env to production will try to connect to the real AWS DynamoDB service. But no where in the documentation I can find what properties to how to setup Para to connect to the AWS DynamoDB service. Can someone help point me on where to find the details or how to go about setting it up? Thank you.
Alex Bogdanovski
@albogdano
@lsornana Para uses the official AWS SDK so you can pass access key and secret through the environment variables or system properties. see the docs on AWS
Lakshman
@lsornana

Thanks @albogdano. So that would be adding the AWS access key and secret into credentials file under the .aws folder right? But what I still don't understand is how to tell Para to use the AWS DynamoDB service instead of using the DynamoDB on the localhost. For example to setup Para to use MongoDB we would be setting the following properties in the application.conf file in the Para server:
⁃ para.dao = "MongoDBDAO"
⁃ para.mongodb.uri = ""
⁃ para.mongodb.host = ""
⁃ para.mongodb.port = ""
⁃ para.mongodb.database = ""
⁃ para.mongodb.user = ""
⁃ para.mongodb.password = ""

Is there a similiar properties we have to set in the application.conf so that Para connected to AWS DynamoDB?

Alex Bogdanovski
@albogdano
@lsornana try setting para.env = "production"
Lakshman
@lsornana
Hi @albogdano. I've already done that setting in the application.conf file and also set the AWS access key/secret. Not sure what I'm doing wrong here to get it to use the AWS DynamoDB. Here is my output when I run Para.
Aug 25 15:18:18 ip-10-0-11-24 java[12319]:      / __ \/ __` / ___/ __` /
Aug 25 15:18:18 ip-10-0-11-24 java[12319]:     / /_/ / /_/ / /  / /_/ /
Aug 25 15:18:18 ip-10-0-11-24 java[12319]:    / .___/\__,_/_/   \__,_/  v1.46.1
Aug 25 15:18:18 ip-10-0-11-24 java[12319]:   /_/
Aug 25 15:18:18 ip-10-0-11-24 java[12319]: 2022-08-25 15:18:18 [INFO ] --- Para.initialize() [production] ---
Aug 25 15:18:18 ip-10-0-11-24 java[12319]: 2022-08-25 15:18:18 [INFO ] Loaded new DAO, Search and Cache implementations - H2DAO, LuceneSearch and CaffeineCache.
Aug 25 15:18:21 ip-10-0-11-24 java[12319]: 2022-08-25 15:18:21 [INFO ] Server is healthy.
Aug 25 15:18:21 ip-10-0-11-24 java[12319]: 2022-08-25 15:18:21 [INFO ] Found root app 'para' and 1 existing child app(s).
Aug 25 15:18:22 ip-10-0-11-24 java[12319]: 2022-08-25 15:18:22 [INFO ] Starting ParaServer using Java 11.0.16 on ip-10-0-11-24 with PID 12319 (/etc/para/para-jar-1.46.1.jar started by root in /etc/para)
Aug 25 15:18:22 ip-10-0-11-24 java[12319]: 2022-08-25 15:18:22 [INFO ] The following 1 profile is active: "production"
Alex Bogdanovski
@albogdano
@lsornana set para.dao = "AWSDynamoDAO"
Lakshman
@lsornana

Thanks @albogdano, it worked now. So running the command java -jar -Dconfig.file=./application.conf -Dloader.path=lib para-jar-1.46.1.jar started up just fine and create the table on AWS DynamoDB. However, when I try to run Para again after a system reboot I get the following error:

software.amazon.awssdk.services.dynamodb.model.DynamoDbException: User: arn:aws:sts::xxxxxxxxxxx:assumed-role/iam-role/i-00000000000000 is not authorized to perform: dynamodb:CreateTable on resource: arn:aws:dynamodb:ap-southeast-2:xxxxxxxxxxx:table/para because no identity-based policy allows the dynamodb:CreateTable action (Service: DynamoDb, Status Code: 400, Request ID: E89KFTGEHK95N9LVP8PHHAO68FVV4KQNSO5AEMVJF66Q9ASUAAJG)

The permissions for the IAM User is correct and worked the first time it ran. Is there something else that I'm missing?

Alex Bogdanovski
@albogdano
@lsornana the access keys used don't allow you to create a table apparently. make sure you have the dynamodb:CreateTable permission
Lakshman
@lsornana
Thanks @albogdano
Lakshman
@lsornana

Hi guys - I've got a questions on setting up AWS S3 storage provider for file uploads. I've setup an IAM User with a AWS access key and secret key. For testing purpose I've attached the AmazonS3FullAccess AWS managed policy. Then created a S3 bucket with the default settings (no bucket policy). When I start the Scoold server I get the following warning message:

S3 bucket *S3_BUCKET_NAME* does not exist. Bucket must be created before files can be uploaded.

So next, I tried adding in a bucket policy allowing all S3 operations but restricted to the IAM User I created by specifying the IAM User ARN for the bucket policy Principal. Still got the same error message.

However, when I set the bucket policy Principal as any user is allowed to perform any S3 operations then it works and I'm able to successfully upload files. Basically it works when the S3 bucket is Public which defeats the purpose of using a access key and shared secret.

Not sure what I'm doing wrong here to get S3 working for file uploads and being secure. Anyone that has going through this before, can help take me through the correct way of setting up and using S3 for file uploads. Thank you.

Alex Bogdanovski
@albogdano
@lsornana perhaps you are in a different region than the bucket? try setting scoold.s3_region (note that this is a Scoold Pro only feature). If you set up file storage in Para, that doesn't mean that Scoold will be able to use it out of the box.
Lakshman
@lsornana

Hi @albogdano - I'm definitely in the same region as the bucket and running Scoold Pro v1.50.1. On the Scoold EC2 instance, I've set the AWS access/secret key in the application.conf file and also in the ~/.aws/credentials file. Here is my Scoold file storage configuration that I've used.

# File Storage Configuration
scoold.s3_bucket = "file-storage-bucket-name"
scoold.s3_path = "uploads"
scoold.s3_region = "ap-southeast-2"
scoold.s3_access_key = "AKIAXXXXXXXXXXXXXXXX"
scoold.s3_secret_key = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

Scoold is only able to find the bucket when it's only set to Public. Which means it's not picking up the access key and secret key from the application.conf or ~/.aws/credentials file.

Alex Bogdanovski
@albogdano
@lsornana are you using Scoold Pro though? File uploads to S3 is a premium feature not available in the open source version of Scoold.
Lakshman
@lsornana
Hi @albogdano - Yes, I'm using Scoold Pro v1.50.1. The file uploads to S3 does work but only when the bucket is set to public.
Alex Bogdanovski
@albogdano
@lsornana I can't reproduce the issue - perhaps there's something wrong with your access keys for S3
Rob Bell
@admin82_gitlab
Hi @albogdano — we're trialling Scoold within our company and we'd need to verify that we can configure Azure AD correctly before we can purchase the Pro version. Are you able to issue us a trial as I understand this feature doesn't exist in the image that's publicly available?
Alex Bogdanovski
@albogdano
@admin82_gitlab LDAP and OAuth 2.0 are available in the open source version of Scoold. SAML is only available in Scoold Pro. You can get started with Azure AD right away using Scoold and keep the same configuration after upgrading to Pro.