These are chat archives for spring-cloud/spring-cloud

19th
May 2017
dragontree101
@dragontree101
May 19 2017 05:28

hey
i use spring-cloud-config 1.3.0.RELEASE pull config from gitlab
has exception

git-upload-pack not permitted

how can i slove this question?

Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.IllegalStateException: Cannot clone or checkout repository] with root cause
[2017-05-19 12:56:00] [lnTCUD] [c1]
[2017-05-19 12:56:00] [lnTCUD] [c1] org.eclipse.jgit.errors.TransportException: http://gitlab.ricebook.net/rhllor/config-common-repository.git: git-upload-pack not permitted
Dave Syer
@dsyer
May 19 2017 05:40
@dragontree101 let me google that for you...
dragontree101
@dragontree101
May 19 2017 05:43
i read this answer, but it invalid, so i question here
Dave Syer
@dsyer
May 19 2017 05:44
That's as much as I know.
dragontree101
@dragontree101
May 19 2017 05:45
ok thanks
Dave Syer
@dsyer
May 19 2017 05:45
@alwaysastudent that's just the way vault works I think. The token is not "exposed to the world" though, it is sent to the config server as a header (over TLS obviously in production)
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 05:46
but to have the token be put into the header the client needs to be configured with it
so it becomes chicken and egg problem
Dave Syer
@dsyer
May 19 2017 05:47
And ever was it thus, with cryptography
I don't know why it was implemented that way actually. Better wait and ask @spencergibb.
Or @mp911de might know if he's around
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 05:48
sure, thank you for the attention
i am just getting started with the vault
Dave Syer
@dsyer
May 19 2017 05:49
It doesn't seem to fit with all the other backends, so I assume there's a reason.
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 05:49
i am researching if something like cloud-foundary have it available as a VCAP service env prop
through some service broker
Dave Syer
@dsyer
May 19 2017 05:51
I think it's part of their SCS service suite
But you have to provide your own Vault
Dave Syer
@dsyer
May 19 2017 05:52
Yeah, that's one way to do it
(provide the Vault, I mean)
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 05:53
is the token we give out in the header a root token ?
Dave Syer
@dsyer
May 19 2017 05:53
I have no idea. I doubt it.
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 05:54
ok, thank you Dave. I hope @spencergibb / @mp991de will chime in with some best practices as well.
Mark Paluch
@mp911de
May 19 2017 06:49
@alwaysastudent static tokens as in config files come at a certain risk and ops complexity
If you're willing to take the risk in exchange for a rather low effort of ops maintenance then there's no issue in general with that approach
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 06:50
I wish the tokens be stored in a secondary GIT repo in an encrypted format. But since if you have got a composite repository going, the X-Config-Token becomes a must have while accessing the config-server.
Mark Paluch
@mp911de
May 19 2017 06:50
If that approach is not appropriate for you/your org, you need a different approach
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 06:51
i have this config server will composite backend of vault+git
I wish, spring.cloud.config.token be given from the centralized git based config where I could store the same in an encrypted manner
Mark Paluch
@mp911de
May 19 2017 06:52
The reason it was built for Config server that way was lowering barriers to entry
You can populate spring.cloud.config.token yourself, i.e if you run a PaaS that obtains a token for your app and injects that into your application
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 06:53
hmm, I am checking this option of binding services if we choose cloud foundry and setting the env var cf set-env
Mark Paluch
@mp911de
May 19 2017 06:53
Storing a token, even if encrypted again comes with secret sprawl and the chicken and egg problem
For CloudFoundry, there are a few options to run Vault
HashiCorp provides their own service broker: https://github.com/hashicorp/cf-vault-service-broker
They create a token and a few mounts (organization/space/application-level) and provide a token for your app to access Vault directly
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 06:55
ok
Mark Paluch
@mp911de
May 19 2017 06:55
This approach gives you a managed token
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 06:56
i see
Mark Paluch
@mp911de
May 19 2017 06:56
I'm currently looking into an integration for Spring apps (Spring Cloud Vault Connector) so your app gets auto-configured to adopt endpoint and authentication
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 06:56
^ is that for the spring-cloud-config ?
or spring-cloud-vault
Mark Paluch
@mp911de
May 19 2017 06:57
Spring Cloud Vault
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 06:57
it is kind of confusing why we these projects are diverged
Mark Paluch
@mp911de
May 19 2017 06:57
Spring Cloud Config with Vault is the lite-variant for using Vault
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 06:57
it makes sense to have the spring-cloud-vault be used by spring-cloud-config since ultimately we are dealing with conifgurations as in the form of secrets
Mark Paluch
@mp911de
May 19 2017 06:58
It depends. Spring Cloud Config with Vault does not allow you generating database credentials properly
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 06:59
and it be better if spring-cloud-config interface the clients to talk to which ever secret backends they chose to use in config server in an opaque manner
Mark Paluch
@mp911de
May 19 2017 06:59
From a client perspective, you don't have the lease notion anymore
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 06:59
ah generating as borrowing live credentials from our app ?
oh that sounds scary
Mark Paluch
@mp911de
May 19 2017 07:00
Yes, generating per-app-instance credentials for your database/messaging system
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 07:00
this will be new to our dba's
how will the app get an elevated sys access to create users in the db ?
Mark Paluch
@mp911de
May 19 2017 07:01
Per-instance credential introduces credential rotation (most organizations create database credentials once in an app lifetime and never change it again)
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 07:01
^ that is right
also dba's hate not to see which app is leaking connection since all share same creds
Mark Paluch
@mp911de
May 19 2017 07:02
It also allows you to trace down from which application instance a particular request originated in case you were compromised
You also can revoke a particular lease (revoke credentials) while other instances can continue operations
Spring Vault/Spring Cloud Vault revoke the credentials on app shutdown

Regarding your question

how will the app get an elevated sys access to create users in the db ?

Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 07:03
i see, for this what infra change we need to bring about ? seems like db systems like oracle needs to adjust
Mark Paluch
@mp911de
May 19 2017 07:04
You set up Vault with a DB user that is able to create users. No one besides the DB admin/Vault operator needs to know these credentials. When you request credentials from Vault, Vault will create a new user with the appropriate grants on your database and return the credentials to your app
IIRC Oracle is not yet supported. Checkout https://www.vaultproject.io/docs/secrets/index.html to see database integrations
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 07:05
ah ok, but this is neat
thanks for educating
Mark Paluch
@mp911de
May 19 2017 07:05
There's an open ticket for Oracle support: hashicorp/vault#2357
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 07:06
but i still feel, spring-cloud-config should bring in these goodness of spring-cloud-vault; since ultimately for secrets configured we would like to reach via same config service ?
or you recommend to just use spring-cloud-vault on clients
Mark Paluch
@mp911de
May 19 2017 07:09
Use Spring Vault/Spring Cloud Vault on clients directly
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 07:09
ok, great.
Mark Paluch
@mp911de
May 19 2017 07:09
Spring Cloud Config is about configuration management. If you need config security without the burden of encryption/key-management, then Spring Vault is the way to go
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 07:10
in spring vault, don't we need an auth mechanism ?
i find token based auth and approle
Mark Paluch
@mp911de
May 19 2017 07:11
Spring Vault supports AWS-EC2, AppId/AppRole, Client Cert, Cubbyhole and static Token auth
Same for Spring Cloud Vault
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 07:11
which one do you recommend that fits well with the above service provider
Mark Paluch
@mp911de
May 19 2017 07:11
It depends on your organization and your security requirements.
If you run on AWS-EC2 this might be a decent option because it does not require too much of setup in the application
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 07:12
nope, we will be on prem
might be evaluating pcf - on prem again
Mark Paluch
@mp911de
May 19 2017 07:12
Some banks and insurances use already client certificate authentication across their apps
Personally, I like cubbyhole because it gives you an indication about security breach (in contrast to static tokens)
If you want to use PCF, then HashiCorp's service broker + token auth is their preferred mechanism. The token lives just as long as the app lives
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 07:13
i see
Mark Paluch
@mp911de
May 19 2017 07:14
(Without the need of storing the token in a config file)
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 07:14
thank you, i saw their VCAP service

The VCAP_SERVICES environment variable will have a section similar to the following:

"vault": [
{
"name": "my-vault",
"plan": "default",
"credentials": {
"address":"https://vault.company.internal:8200/",
"auth":{
"accessor":"b1074bb8-4d15-36cf-54dd-2716fb8ac91d",
"token":"dff95895-6a03-0b29-6458-dc8602dc9df8"
},
"backends":{
"generic":"cf/203f2469-04e4-47b8-bc17-f3af56df8019/secret",
"transit":"cf/203f2469-04e4-47b8-bc17-f3af56df8019/transit"
},
"backends_shared":{
"organization":"cf/3c88c61e-875b-4530-b269-970f926340c4/secret",
"space":"cf/0348d384-f7d4-462b-9bd9-5a4c05b21b6c/secret"
}
}
}
]

and each app would get a unique token ?
Mark Paluch
@mp911de
May 19 2017 07:15
Tokens are app-specific. I don't think they are instance-specific.
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 07:16
i see
Mark Paluch
@mp911de
May 19 2017 07:16
However, using Vault in anyway is a major improvement to today's ops practices in most organizations.
I gotta run, hope I could provide you with some insights.
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 07:17
You were tremendous
I am glad I was staying online late. been struggling to get my head around these things and you came online. It was very helpful. Thank you again.
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 08:57
hi @mp911de - I was able to set up the service broker, register the same and bind to an app.
it generates me a token from my vault service running on an ec2
and also creates this mounts
    "credentials": {
     "address": "https://blah.amazonaws.com:8200/",
     "auth": {
      "accessor": "25afc1eb-5ef7-1cbf-2aca-62468ac913af",
      "token": "d3a9123e-5c19-7562-432e-ebb763bae079"
     },
     "backends": {
      "generic": "cf/7abe1cfa-4a17-407a-8bf9-09f1544951c7/secret",
      "transit": "cf/7abe1cfa-4a17-407a-8bf9-09f1544951c7/transit"
     },
     "backends_shared": {
      "organization": "cf/9a9fb216-42ae-4bfb-865e-69b44e256a98/secret",
      "space": "cf/d0cae1cd-e4e2-44f4-9718-a1f6d54d605b/secret"  }
it looks pretty weird since it is very random
the token it generates has access only in that mount
atleast that is how I could understand or see it work
and we want to store secrets on a well known mount right? if this is generated random and have the token scoped to that mount alone, it is basically useless
i am not sure, if you have any exposure to this. If so kindly on your spare, write me back,
thank you
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 09:15
I think i get it now that these random ids on the mounts are in fact guids
cf org my-org --guid
9a9fb216-42ae-4bfb-865e-69b44e256a98


cf space my-space --guid
d0cae1cd-e4e2-44f4-9718-a1f6d54d605b
it seems like a weird way to do manage this
hmm
Mark Paluch
@mp911de
May 19 2017 10:12
The broker uses app/space/org guids to create dedicated mounts
Matt Day
@mattsday
May 19 2017 10:34
Hi there, I'm looking for the right pattern on Cloud Foundry with Eureka, Zuul and blue/green deployments with no downtime. Is there guidance on best-practise to migrate from v1 to v2? At the moment, I lose traffic as green goes down but is still in Eureka.
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 16:23
@mp911de - thank you. Sidestepping a bit, but feel it is a bit relevant. I believe spring cloud connectors project would help realizing bounded services as Spring beans on cloud foundry. Typically when we create a cf service, say a user provided sql service, we have to pass the credentials . Can this be picked from or mapped to some secret stored in the vault, which would be pulled at run time ? Or if we stretch that bit can spring-cloud-vault be made to work in conjunction with vault and the sql service broker to get on demand per instance credentials - this seems like a custom broker, eh ?
Karthik Mahadeva Iyer
@alwaysastudent
May 19 2017 16:36
just would like to know if it practical to get the credentials from vault for a bound service, using spring-cloud-connector ?
Mark Paluch
@mp911de
May 19 2017 20:51
Pivotal is working on an improvement for credentials but I'm not involved too much into PCF