hi @matteofigus so regarding my last question about S3 bucket policies, it works now but only if I partly allow public access to the S3 bucket by checking only 1 and 3 (see screenshot). Were you able to publish and work with the registry when you have full "Block public access" enabled and just grant access to the certain user ?
Now we can read and write from the bucket using IAM role with correct policy, but for some reason I can't publish to the registry when I'm using S3 keys and secrets for a user :
getting error
Now we can read and write from the bucket using IAM role with correct policy, but for some reason I can't publish to the registry when I'm using S3 keys and secrets for a user :
getting error
error occurred when publishing to a registry {}
It's only one thing that prevents us from running full CI/CD and use OC in production(
@matteofigus thank you for your help again, now it make sense. I thought that I'm doing something wrong and there is a way to make it work with "block public access" enabled. We are using s3, k8s and docker to run registry and security people concerned about making bucket public...
But you may need to change some code in the registry to avoid saving with public ACL..which shouldn’t be too much effort
You can fork the storage adapter and override it making sure all gets private here https://github.com/opencomponents/storage-adapters/blob/f164e7fefa1e1a874b28c8612e38f988a090b30b/packages/oc-s3-storage-adapter/index.js#L212
@matteofigus Hi! We use SSR only and I think I'll need to use you suggestion and override public part. So after in my server.js I should have something like this when I use storage adapter?
const oc = require('oc');
const s3 = require('oc-s3-storage-adapter');
let config = {
baseUrl: 'my url',
port: 3000,
refreshInterval: 600,
pollingInterval: 5,
templates: [require('oc-template-jade')],
storage: {
adapter: s3,
options: {
key: 'my key',
secret: 'my secret',
bucket: 'my bucket name',
region: 'us-east-2',
componentsDir: 'components',
sslEnabled: false,
s3ForcePathStyle: true,
path - > ??
debug: true,
endpoint: 'http://localhost:8080'
}
},
env: { name: 'production' }
};
let registry = new oc.Registry(config);
registry.start(function(err, app){
if(err) {
console.log('Registry not started: ', err);
process.exit(1);
}
});I was looking for some documentation for storage adapters, maybe you have some examples? The most part is self explanatory, but maybe I'll miss something accidentally or I need more configuration under 'options'...
@alex all options are passed to the s3 adapter as the conf object. https://github.com/opencomponents/storage-adapters/blob/master/packages/oc-s3-storage-adapter/index.js#L17
@matteofigus @kmcrawford Hi guys, thank you so much for your help and suggestions, I made adjustments to the adapter and I'm able to work with fully private bucket & SSR. In order to do that we sacrifice components preview, but it is ok in our case.
The only thing that I want to optimize is a mechanism of getting components from the registry with private S3 bucket, now I have to send SECRET and KEY along with request(otherwise getting Access denied) to a registry trying to combine it with
Have a great day !
The only thing that I want to optimize is a mechanism of getting components from the registry with private S3 bucket, now I have to send SECRET and KEY along with request(otherwise getting Access denied) to a registry trying to combine it with
client.renderComponents() but it makes rendering a little slower, so I hope I'm on the right track because I don't have any other option in my mind.Have a great day !
@aresobus_twitter check out https://github.com/crunchyroll/evs-s3helper I haven’t used it but it should be able to proxy a private s3 bucket on an internal server
@kmcrawford wow thank you! I'm trying to do something like that in NodeJS and express.js router, so far can't get through access denied message.
My initial idea was to do following:
- get a pre-signed url's:
And then pass it to the client = to have something like that:const AWS = require('aws-sdk') const s3 = new AWS.S3() AWS.config.update({accessKeyId: 'id', secretAccessKey: 'key'}) const myBucket = 'bucket-name' const signedUrlExpireSeconds = 60 * 5 <-- just a test const url = s3.getSignedUrl('getObject', { Bucket: myBucket, Key: "/", Expires: signedUrlExpireSeconds }) console.log(url)
const client = new Client({
registries: {
serverRendering: pre-signed url here <---
},
})maybe I'm on the wrong track, will see
I guess an easier way would be to use IAM if the consumer is running on aws too, so you don’t need to pass creds around which is risky and fragile. If the consumer is outside, you need creds. The only other strategy for rendering SSR without fetching view etc is asking the registry to do the render for you. When you make the rendering call, you can pass an accept header to ask for rendered result and the client should be able to handle the result. But obviously this is less optimal because you are overloading the registry with more compute and you are not caching the views on the clients impacting performance.
Try passing as header Accept: application/vnd.oc.rendered+json
@matteofigus I was looking at this conversation earlier:
opencomponents/oc#324
so I'm doing almost everything in the same way, except Accept header, will add that. The problem is that I can't go pass this step with fully private bucket to get a response:
opencomponents/oc#324
so I'm doing almost everything in the same way, except Accept header, will add that. The problem is that I can't go pass this step with fully private bucket to get a response:
POST https://oc-registry.com -H "Accept: application/vnd.oc.info+json"
{
components: [{
name: 'component1',
version: '1.X.X'
}, {
name: 'component2',
version: '~1.2.4'
}]
}
but I think it's because I don't have enough experience with AWS and I'll find what is wrong
I need a credentials because consumer is outside ;(
hi all :wave: im getting into OC. i setup a local registry like this:
now if i open a new terminal and try
but i do see the request in the registry window:
is this a bug in the code or between keyboard and chair?
oc dev ../components 3000 and put my test component in the components dir.now if i open a new terminal and try
oc registry add http://localhost:3000 i get this error:not a valid oc registrybut i do see the request in the registry window:
GET / 200 0.514 ms - 160is this a bug in the code or between keyboard and chair?
1 reply
After you setup a registry you can add that url and then publish to it
No I’m afraid there isn’t. You can check the storage adapters where you can have various options outside of aws but all of them somehow need some storage. But the riak may be something you can investigate
Hello I am trying to understand how Registry Rest API works . Tried to follow the documentation at https://github.com/opencomponents/oc/wiki/Registry . I am bit puzzled about how the registry will work if I want to host the registry in a AWS ECS (EC2) instance.
oc init hello-world throwing error -bash: oc: command not found
https://github.com/opencomponents/oc/wiki I am following this instructions
Hello I am trying to understand how Registry Rest API works . Tried to follow the documentation at https://github.com/opencomponents/oc/wiki/Registry . I am bit puzzled about how the registry will work if I want to host the registry in a AWS ECS (EC2) instance.
@shinup after deploying the registry, you should have a base url you can use with the CLI. An easy way to get started with AWS is to use docker for the registry and use Elastic Beanstalk - an example of a container https://github.com/ciricihq/oc-docker
@yugandhar-pathi are you looking for something like this? https://github.com/opencomponents/react-oc
Have a look at this one - it's the most updated one https://github.com/opencomponents/oc-template-react
any way to get more information on whats going on? Followed this Heroku guide step by step and heroku outputting this
2021-03-15T20:23:45.713168+00:00 app[web.1]: > opencomponents-starter-kit@1.0.0 start /app
2021-03-15T20:23:45.713169+00:00 app[web.1]: > node server.js
2021-03-15T20:23:45.713169+00:00 app[web.1]:
2021-03-15T20:23:49.694652+00:00 app[web.1]: Registry not started: components_list_save
2021-03-15T20:23:49.842166+00:00 app[web.1]: npm ERR! code ELIFECYCLE
2021-03-15T20:23:49.844844+00:00 app[web.1]: npm ERR! errno 1
2021-03-15T20:23:49.887172+00:00 app[web.1]: npm ERR! opencomponents-starter-kit@1.0.0 start:node server.js`