These are chat archives for spring-cloud/spring-cloud

29th
Jun 2016
bitsofinfo
@bitsofinfo
Jun 29 2016 01:40
@marcosbarbero k, so if some of the configs for the "config server" in git are {cipher} prefixed, where do the encrypt.keyStore.* properties get defined? I tried them in bootstrap.yml but it doesn't seem to take, just unclear what the minimum set of properties are required local to the app, before it pulls from git then consumes the rest from there for self config
Marcos Barbero
@marcosbarbero
Jun 29 2016 02:14
@bitsofinfo you can put the encrypt keys in git repository as well. For ciphering you must have JCE installed on the config server machine
bitsofinfo
@bitsofinfo
Jun 29 2016 02:23

Hmm, well, a bit lost I guess.

So locally in my project i have a bootstrap.yml with

spring.cloud.config.server.bootstrap: true
spring.application.name: my-config-server
spring.cloud.config.server.git.uri: ${my.git.url}

I start the app with:

java  \
-Dmy.git.url="file:///path/to/gitrepo" \
-Dmy.keystore.store_pass="123" \
-Dmy.keystore.default_key_alias=mykey \
-Dmy.keystore.default_key_secret="123" \
-Dmy.keystore.resource_location="classpath:/keystore.jks" \
-jar myapp.jar

In file:///path/to/gitrepo I have:

my-config-server.yml with:

encrypt:
  keyStore:
    location: ${my.keystore.resource_location}
    password: ${my.keystore.store_pass}
    alias: ${my.keystore.default_key_alias}
    secret: ${my.keystore.default_key_secret}

my.custom.prop: '{cipher}xxxxcryptedtext'

The result when booting is that I see it pulling the configs from the git repo, but it does not decrypt the ciphered configs. Its like its not getting the correct info for the encrypt.keystore props

Note the above all works fine, this is all just in application.yml on the config server, and spring.cloud.config.server.bootstrap: false. I would just like to get the config-servers "config" itself in git as well rather than embedded statically in the project/jar
bitsofinfo
@bitsofinfo
Jun 29 2016 02:30
note: the keystore.jks is in the jar
Dave Syer
@dsyer
Jun 29 2016 02:31
It's too late to put the keystore config in the config server repo. The values you have there serve no purpose in a git repo anyway. If you put them in bootstrap.yml (as @marcosbarbero advised) I imagine it would work.
bitsofinfo
@bitsofinfo
Jun 29 2016 02:36

agreed, I have no desire to have any of the keystore info in git. I've already tried a bootstrap.yml that looks like this and it still does not work. Was hoping bootstrap.yml could have the minimum to load the keys for crypto, connect to git then pull the rest of its config from my-config-server.yml in git.

spring.cloud.config.server.bootstrap: true
spring.application.name: my-config-server
spring.cloud.config.server.git.uri: ${my.git.url}

encrypt:
  keyStore:
    location: ${my.keystore.resource_location}
    password: ${my.keystore.store_pass}
    alias: ${my.keystore.default_key_alias}
    secret: ${my.keystore.default_key_secret}

When starting the app with this, I just end up with

java.lang.IllegalStateException: Cannot decrypt: key=my.custom.prop
Dave Syer
@dsyer
Jun 29 2016 02:37
At least it tried.
Some more detail on the stack trace might help
Maybe it's the wrong key?
Maybe you don't have the JCE extensions?
bitsofinfo
@bitsofinfo
Jun 29 2016 02:38
nada, this all works if all the config server's "configs" are local (not in git)
so not a bad key etc
bitsofinfo
@bitsofinfo
Jun 29 2016 02:44
the encrypted value I have in this config file, can be decrypted by the config server fine, if the config-server boots w/ all locally sourced configs (all inputs -D to jar) are same each run (just bootstrap.yml and the existance/or not of locally sourced confs, vs being in git)
Dave Syer
@dsyer
Jun 29 2016 02:46
Works for me
bitsofinfo
@bitsofinfo
Jun 29 2016 02:46
k, then something on my end I'm not seeing, investigating further
What version of config server are you using?
I can push my changes to a branch if it helps
(to the sample I mean)
bitsofinfo
@bitsofinfo
Jun 29 2016 02:48
whatever is bound to org.springframework.cloud:spring-cloud-dependencies:Brixton.SR1 which I assume would be the latest stable
1.1.1
the sample at that url does not have encrypt props in it, i assume that is what you are referring to, that you must be testing locally (added)
Dave Syer
@dsyer
Jun 29 2016 02:52
Yes
+          uri: file://${HOME}/dev/demo/config-repo
+encrypt:
+  failOnError: false
+  keyStore:
+    location: classpath:keystore.jks
+    password: ${KEYSTORE_PASSWORD:foobar} # don't use a default in production
+    alias: test
That's my one change
Plus adding the keystore to that location
bitsofinfo
@bitsofinfo
Jun 29 2016 02:53
k, ok, so got to be something else on my end. looking, thanks
Dave Syer
@dsyer
Jun 29 2016 02:53
and adding a {cipher} property at configserver.yml in git repo
bitsofinfo
@bitsofinfo
Jun 29 2016 02:53
yep
bitsofinfo
@bitsofinfo
Jun 29 2016 03:09
ok, well here appears to be the diff, my {cipher} is postfixed with a {key:keyname}
Without that {key:keyname} it works (the keyname is legit)
Dave Syer
@dsyer
Jun 29 2016 03:12
{key:keyname} implies you have a custom text encryptor locator?
Is it in your bootstrap context?
bitsofinfo
@bitsofinfo
Jun 29 2016 03:14
hmmm no. The way i read the docs "the default locator will look for keys in the store with aliases as supplied by the "key" prefix, i.e. with a cipher text like this" I understood it that {key:aliasname} meant that it would attempt to decrupt the given {crypt} line with a different aliased key in the store other than the default one you specified...
Dave Syer
@dsyer
Jun 29 2016 03:16
Yeah, but the default locator might not be in the bootstrap context
I don't see how it would be
bitsofinfo
@bitsofinfo
Jun 29 2016 03:17
I can get by without they {key:...} postfix, however it would be nice, then I know what key that cipher text is bound to over time
Dave Syer
@dsyer
Jun 29 2016 03:17
but I guess it could be
You can add it to your bootstrap context
EncryptionAutoConfiguration is not a org.springframework.cloud.bootstrap.BootstrapConfiguration by default
But I think you could just add it to a spring.factories and it might work
bitsofinfo
@bitsofinfo
Jun 29 2016 03:21
so are you saying in spring.factories add a line like ``org.springframework.boot.autoconfigure.EnableAutoConfiguration=org.springframework.cloud.config.server.config. EncryptionAutoConfiguration ?
or like: org.springframework.cloud.bootstrap.BootstrapConfiguration=org.springframework.cloud.config.server.config.EncryptionAutoConfiguration
suspect the latter
Spencer Gibb
@spencergibb
Jun 29 2016 03:24
Yes, latter
Dave Syer
@dsyer
Jun 29 2016 03:25
Yep
If that works we can make it the default when spring.cloud.config.server.bootstrap=true
bitsofinfo
@bitsofinfo
Jun 29 2016 03:26
boom, hell yes! thanks guys you rock. Saved me a ton of time. Works
Dave Syer
@dsyer
Jun 29 2016 03:26
Can you open an issue in github?
bitsofinfo
@bitsofinfo
Jun 29 2016 03:26
yes i will, i imagine someone else could hit this
Dave Syer
@dsyer
Jun 29 2016 03:33
thanks
bitsofinfo
@bitsofinfo
Jun 29 2016 03:33
spring-cloud/spring-cloud-config#434
thanks again guys!
bitsofinfo
@bitsofinfo
Jun 29 2016 05:10
@dsyer did you validate that the value yielded by what you did was actually decrypted? https://gitter.im/spring-cloud/spring-cloud?at=577338371ac8bd1a4d893a2e
Dave Syer
@dsyer
Jun 29 2016 10:47
Yes
pradeepkusingh
@pradeepkusingh
Jun 29 2016 14:41
@dsyer : I have a requirement that if circuit braker is opened don' send request. I am trying to find API which can tell me the ircuit is opend for primary and fall back method. Can you please help me ?
Dave Syer
@dsyer
Jun 29 2016 15:27
Isn't that the whole point of a circuit breaker?
It should be transparent to application code
AFAIK you can only tell which circuits are open by looking at the metrics
But application code shouldn't need to do that
pradeepkusingh
@pradeepkusingh
Jun 29 2016 15:53
ok..i agree.. sometime people design app create more complexitiy, I think I am one of them :)
bitsofinfo
@bitsofinfo
Jun 29 2016 18:38
@dsyer in relation to spring-cloud/spring-cloud-config#434 I created a runnable example of the problem I am still facing. Thoughts? https://github.com/bitsofinfo/test-config-server-crypto
I need the decrypted value for another bean I want to create during the boot, but its like the decrypted value not available at that point. Only everything trailing {cipher} Confused
bitsofinfo
@bitsofinfo
Jun 29 2016 19:14
even doing context.getEnvironment().getProperty("test.crypted.prop") after the context is booted does not yield the decrypted value, its only accessible in the clear via config request calls to http://localhost:8888/config-server-test1/test
bitsofinfo
@bitsofinfo
Jun 29 2016 20:32
anyone else on here have any thoughts on the above?
Dave Syer
@dsyer
Jun 29 2016 20:55
Yes, there is an order in the lifecycle for everything.
You are trying to access the data before it is available.
bitsofinfo
@bitsofinfo
Jun 29 2016 20:55
yeah figured something like that.
I guess I don't get why they are not decrypted even after the context is fully loaded. (at least in this test app I created) I just have a need to secure some of the configs (crypted) for the config-server itself
Dave Syer
@dsyer
Jun 29 2016 20:58
Once it is fully loaded the decrypted value should be available
There's a chicken and egg problem though, as I'm sure you can appreciate.
bitsofinfo
@bitsofinfo
Jun 29 2016 20:59
ya I can imagine, just trying to figure out how to make it work.
Dave Syer
@dsyer
Jun 29 2016 20:59
In the sample app you aren't doing anything custom in the bootstrap (except decryption as discussed)
bitsofinfo
@bitsofinfo
Jun 29 2016 20:59
This message was deleted
Dave Syer
@dsyer
Jun 29 2016 20:59
So I'm not 100% clear yet what isn't working
(You need new lines after the fences)
bitsofinfo
@bitsofinfo
Jun 29 2016 21:00
public static void main(String[] args) throws Exception {
       ApplicationContext context = SpringApplication.run(new Class[]{Application.class}, args);

       System.out.println(env.getProperty("test.clear.prop"));
       System.out.println(env.getProperty("test.crypted.prop"));

       System.out.println(context.getEnvironment().getProperty("test.crypted.prop"));


    }
there we go
Dave Syer
@dsyer
Jun 29 2016 21:01
So env is a static field you set to the value injected via @Autowired.
bitsofinfo
@bitsofinfo
Jun 29 2016 21:01
definately working via GETs through the config rest interface though
Dave Syer
@dsyer
Jun 29 2016 21:01
and it doesn't see the decryption?
But the second one does (via context.getEnvironment())?
bitsofinfo
@bitsofinfo
Jun 29 2016 21:01
the only place I can get a decrypted value is via calls to localhost:8888/config-server-test1/n etc
Dave Syer
@dsyer
Jun 29 2016 21:02
Hmm
OK let me try it
bitsofinfo
@bitsofinfo
Jun 29 2016 21:02
when I try to print them out in that sample app it always prints out the crypted stuff AFTER {cipher}
so something is evaluating the value, its just not decrypting it at that point
maybe something in that environment impl is pointing to a stale ref of something pre-crypto eval under this scenario
Dave Syer
@dsyer
Jun 29 2016 21:07
I lied to you yesterday.
I was looking at the wrong thing.
So your app is the same as mine now.
bitsofinfo
@bitsofinfo
Jun 29 2016 21:10
ok, not totally following
Dave Syer
@dsyer
Jun 29 2016 21:10
Sorry.
I'm just apologising (badly)
bitsofinfo
@bitsofinfo
Jun 29 2016 21:10
ha
Dave Syer
@dsyer
Jun 29 2016 21:10
I'll look at it again
It seems the workaround we came up with yesterday is not working after all
It'll be the order
bitsofinfo
@bitsofinfo
Jun 29 2016 21:11
what "worked" for me yesterday was fixed by what you recommended, but its just this particular issue I tried today that is new (decrypting values for the config-server needs to wire beans itself)
Dave Syer
@dsyer
Jun 29 2016 21:17
Yeah
OK so it's a new feature
bitsofinfo
@bitsofinfo
Jun 29 2016 21:18
how extensive of a change is it to make this work?
Dave Syer
@dsyer
Jun 29 2016 21:19
Fairly
The TextEncryptorLocatorstrategy is only used at the minute to feed an EnvironmentEncryptor
and that's not used to decrypt the application context Environment (it applies only to the Cloud Config Server Environment)
We could do it I think
If you want to have a go, look at EncryptionBootstrapConfigurationand notice that it uses a raw TextEncryptor instead of delegating to the TextEncryptorLocatorstrategy.
We'd have to change it to do the delegation
bitsofinfo
@bitsofinfo
Jun 29 2016 21:22
k, so the props for the config-server itself are available in both Environments, (config server proxied one and the appcontext itself) but just no decryptor bound to the latter
Dave Syer
@dsyer
Jun 29 2016 21:22
Yes
I have to go to a meeting now
Back later
bitsofinfo
@bitsofinfo
Jun 29 2016 21:22
k, I will take a look. Right now just coming up w/ a workaround to get me going.
Dave Syer
@dsyer
Jun 29 2016 22:08
I might have a workaround
2016-06-29 15:08:13.880  INFO 6239 --- [           main] org.bitsofinfo.app1.Application          : Started Application in 15.522 seconds (JVM running for 15.937)
overidden
123
123
That's the correct output, right?
bitsofinfo
@bitsofinfo
Jun 29 2016 22:10
looks good
Dave Syer
@dsyer
Jun 29 2016 22:10
I can push my change to your repo if you make me a collaborator
It's a bit of a hack, but you could tidy it up by just copying a package private utility class from Spring Cloud.
added u, will check it out when I get back, thx for looking into it.
Dave Syer
@dsyer
Jun 29 2016 22:18
To git@github.com:bitsofinfo/test-config-server-crypto.git
 * [new branch]      workaround ->
arca1n
@arca1n
Jun 29 2016 22:26
I am spring-boot n00b. What would be a recommended way to implementing a message queue backed by redis? I am considering spring-cloud-stream with the redis binder
Dave Syer
@dsyer
Jun 29 2016 23:02
But note that redis is not really a messaging platform
We don't recommend using it in production
I guess that would be the recommended way then