Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Mar 31 08:37
    matthewdavidson removed as member
  • Mar 12 14:23
    matteofigus commented #1184
  • Mar 12 14:23
    matteofigus commented #1184
  • Mar 12 14:11
    dianatamas commented #1184
  • Mar 12 14:11
    dianatamas commented #1184
  • Mar 12 14:10
    dianatamas updated the wiki
  • Mar 12 14:10
    dianatamas updated the wiki
  • Mar 12 14:09
    dianatamas updated the wiki
  • Mar 12 14:09
    dianatamas updated the wiki
  • Mar 12 12:24

    matteofigus on master

    0.48.18 changelog (compare)

  • Mar 12 12:24

    matteofigus on master

    0.48.18 changelog (compare)

  • Mar 12 12:24

    matteofigus on v0.48.18

    (compare)

  • Mar 12 12:24

    matteofigus on v0.48.18

    (compare)

  • Mar 12 12:22
    matteofigus commented #1184
  • Mar 12 12:22
    matteofigus commented #1184
  • Mar 12 12:20

    matteofigus on master

    Add support for custom keepAliv… Merge pull request #1184 from d… (compare)

  • Mar 12 12:20

    matteofigus on master

    Add support for custom keepAliv… Merge pull request #1184 from d… (compare)

  • Mar 12 12:20
    matteofigus closed #1184
  • Mar 12 12:20
    matteofigus closed #1184
  • Mar 12 11:35
    dianatamas opened #1184
Matteo Figus
@matteofigus
@aresobus_twitter the registry needs to be able to write to s3 as it uses it to store various metadata including the components list. From the error it looks like the credentials used don’t have write access to that bucket.
The bucket shouldn’t need to be public, it just needs to be writeable by the registry by the user you specified the credentials for
ALex
@aresobus_twitter

@matteofigus wow ;) I didn't expect to receive immediate answer from you, thank you!

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "OurSid",
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::bucket-name",
                "arn:aws:s3:::bucket-name/*"
            ]
        }
    ]
}

Hmm, weird. Above you can see the policy for a user so he can perform every action with this bucket
I tried manually write to the bucket using this credentials and it works, but during the deployment registry won't start and still getting this error (( so you think issue is with access and there is no need for additional configuration ?
I apologize for very theoretical questions ;)

Matteo Figus
@matteofigus
Uhm the policy actually looks ok 👍
Can you check the bucket policy too?
Wondering if there is something strange going on there
ALex
@aresobus_twitter
@matteofigus will do, do we want the bucket to be public or just allow this user read and write to the bucket?
I tried deploying the same config to my personal AWS account via heroku and it works and it makes me think that some corporate policy configuration prevents it from working...
ALex
@aresobus_twitter

@matteofigus here is bucket policy(allow all actions for user above)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "statement1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::account-id:user/user-name"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::our-bucket-name"
        },
        {
            "Sid": "statement2",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::account-id:user/user-name"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::our-bucket-name/*"
        }
    ]
}

Is something looks weird to you ?

note: bucket is not public as I mentioned earlier
Matteo Figus
@matteofigus
The policy looks ok. The registry doesn’t need the bucket to be public but it puts some objects with public acls - i wonder if that’s the bit that is somehow conflicting with some existing account policies
But on registry start, all the components metadata is private, so it should be ok..
ALex
@aresobus_twitter
@matteofigus thanks a lot, will try to play around with policies, at least I know what is wrong. Have a great weekend!
ALex
@aresobus_twitter
hi @matteofigus so regarding my last question about S3 bucket policies, it works now but only if I partly allow public access to the S3 bucket by checking only 1 and 3 (see screenshot). Were you able to publish and work with the registry when you have full "Block public access" enabled and just grant access to the certain user ?
Now we can read and write from the bucket using IAM role with correct policy, but for some reason I can't publish to the registry when I'm using S3 keys and secrets for a user :
getting error error occurred when publishing to a registry {}
Screen Shot 2020-09-10 at 6.32.32 PM.png
It's only one thing that prevents us from running full CI/CD and use OC in production(
Matteo Figus
@matteofigus
oh ok, that makes sense, ok, I didn't think of the block policies. Some objects such as the package.json and the static assets are published with public ACL as are needed by the browser, so they need to be public. The server.js is instead published as private
Matteo Figus
@matteofigus
@aresobus_twitter out of curiosity, are you running the registry inside AWS? In that case, you can probably just omit key/secret from the registry configuration and infer that via IAM
ALex
@aresobus_twitter
@matteofigus thank you for your help again, now it make sense. I thought that I'm doing something wrong and there is a way to make it work with "block public access" enabled. We are using s3, k8s and docker to run registry and security people concerned about making bucket public...
Matteo Figus
@matteofigus
Ok. Consider that you need some stuff to be public if you need client side rendering and for static assets..but if you do SSR only and you setup a CDN on top of s3 with the proper policies to go public, then you may be ok with full private.
But you may need to change some code in the registry to avoid saving with public ACL..which shouldn’t be too much effort
ALex
@aresobus_twitter

@matteofigus Hi! We use SSR only and I think I'll need to use you suggestion and override public part. So after in my server.js I should have something like this when I use storage adapter?

const oc = require('oc');
const s3 = require('oc-s3-storage-adapter');

let config = {
  baseUrl: 'my url', 
  port: 3000,
  refreshInterval: 600,
  pollingInterval: 5,
  templates: [require('oc-template-jade')],
  storage: {
    adapter: s3,
    options: {
      key: 'my key', 
      secret: 'my secret', 
      bucket: 'my bucket name',
      region: 'us-east-2',
      componentsDir: 'components',
      sslEnabled: false,
      s3ForcePathStyle: true,
      path - > ??
      debug: true,
      endpoint: 'http://localhost:8080'
    }    
  },
  env: { name: 'production' }
};

let registry = new oc.Registry(config);

registry.start(function(err, app){
  if(err) {
    console.log('Registry not started: ', err);
    process.exit(1);
  }
});

I was looking for some documentation for storage adapters, maybe you have some examples? The most part is self explanatory, but maybe I'll miss something accidentally or I need more configuration under 'options'...

Ken Crawford
@kmcrawford
ALex
@aresobus_twitter
@kmcrawford thank you !
ALex
@aresobus_twitter
@matteofigus @kmcrawford Hi guys, thank you so much for your help and suggestions, I made adjustments to the adapter and I'm able to work with fully private bucket & SSR. In order to do that we sacrifice components preview, but it is ok in our case.
The only thing that I want to optimize is a mechanism of getting components from the registry with private S3 bucket, now I have to send SECRET and KEY along with request(otherwise getting Access denied) to a registry trying to combine it with
client.renderComponents() but it makes rendering a little slower, so I hope I'm on the right track because I don't have any other option in my mind.
Have a great day !
Ken Crawford
@kmcrawford
:thumbsup:
Ken Crawford
@kmcrawford
@aresobus_twitter check out https://github.com/crunchyroll/evs-s3helper I haven’t used it but it should be able to proxy a private s3 bucket on an internal server
ALex
@aresobus_twitter

@kmcrawford wow thank you! I'm trying to do something like that in NodeJS and express.js router, so far can't get through access denied message.
My initial idea was to do following:

  1. get a pre-signed url's:
    const AWS = require('aws-sdk')
    const s3 = new AWS.S3()
    AWS.config.update({accessKeyId: 'id', secretAccessKey: 'key'})
    const myBucket = 'bucket-name'
    const signedUrlExpireSeconds = 60 * 5 <-- just a test
    const url = s3.getSignedUrl('getObject', {
     Bucket: myBucket,
     Key: "/",
     Expires: signedUrlExpireSeconds
    })
    console.log(url)
    And then pass it to the client = to have something like that:
const client = new Client({
    registries: {
      serverRendering: pre-signed url here <---

    },
  })

maybe I'm on the wrong track, will see

Matteo Figus
@matteofigus
I guess an easier way would be to use IAM if the consumer is running on aws too, so you don’t need to pass creds around which is risky and fragile. If the consumer is outside, you need creds. The only other strategy for rendering SSR without fetching view etc is asking the registry to do the render for you. When you make the rendering call, you can pass an accept header to ask for rendered result and the client should be able to handle the result. But obviously this is less optimal because you are overloading the registry with more compute and you are not caching the views on the clients impacting performance.
Try passing as header Accept: application/vnd.oc.rendered+json
ALex
@aresobus_twitter
@matteofigus I was looking at this conversation earlier:
opencomponents/oc#324
so I'm doing almost everything in the same way, except Accept header, will add that. The problem is that I can't go pass this step with fully private bucket to get a response:
POST https://oc-registry.com -H "Accept: application/vnd.oc.info+json"
{
  components: [{
    name: 'component1',
    version: '1.X.X'
  }, {
    name: 'component2',
    version: '~1.2.4'
  }]
}
but I think it's because I don't have enough experience with AWS and I'll find what is wrong
I need a credentials because consumer is outside ;(
Artëm Tsvetkov
@adeptex
hi all :wave: im getting into OC. i setup a local registry like this: oc dev ../components 3000 and put my test component in the components dir.
now if i open a new terminal and try oc registry add http://localhost:3000 i get this error:
not a valid oc registry
but i do see the request in the registry window:
GET / 200 0.514 ms - 160
is this a bug in the code or between keyboard and chair?
1 reply
Matteo Figus
@matteofigus
Hi Artem, the add command is used for setting a publish registry instance as target. When you run the oc dev you are running a local mock therefore that cannot be a publish target and the cli detects that.
1 reply
After you setup a registry you can add that url and then publish to it
No I’m afraid there isn’t. You can check the storage adapters where you can have various options outside of aws but all of them somehow need some storage. But the riak may be something you can investigate
ALex
@aresobus_twitter
@matteofigus @kmcrawford thank you for help earlier, I forget to mention that everything worked out and I found a way to make OC work with fully private bucket and now we have it running in PROD!
Ken Crawford
@kmcrawford
👍🏻
Matteo Figus
@matteofigus
Very nice!
Shinu Pushpan
@shinup
Hello I am trying to understand how Registry Rest API works . Tried to follow the documentation at https://github.com/opencomponents/oc/wiki/Registry . I am bit puzzled about how the registry will work if I want to host the registry in a AWS ECS (EC2) instance.
playforada97
@playforada97
Hi I am setting up OC project
oc init hello-world throwing error -bash: oc: command not found
https://github.com/opencomponents/oc/wiki I am following this instructions
ALex
@aresobus_twitter
@playforada97 run ```npm install -g oc```` it looks like you don't have framework installed
Matteo Figus
@matteofigus

Hello I am trying to understand how Registry Rest API works . Tried to follow the documentation at https://github.com/opencomponents/oc/wiki/Registry . I am bit puzzled about how the registry will work if I want to host the registry in a AWS ECS (EC2) instance.

@shinup after deploying the registry, you should have a base url you can use with the CLI. An easy way to get started with AWS is to use docker for the registry and use Elastic Beanstalk - an example of a container https://github.com/ciricihq/oc-docker

yugandhar-pathi
@yugandhar-pathi
@matteofigus i am new to open components .... after spending sometime i understand that it helps us to stitch micro front ends together ..is my understanding correct ?
Matteo Figus
@matteofigus
Yes
Ken Crawford
@kmcrawford
@matteofigus can I get a publish of storage-adapters?