by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Aug 11 16:11
    jeffwecan commented #620
  • Aug 11 15:30
    Travis hvac/hvac#621 broken (1749)
  • Aug 11 15:23
    Borg-Can-Code commented #620
  • Aug 11 15:22
    Borg-Can-Code edited #621
  • Aug 11 15:18
    Borg-Can-Code opened #621
  • Aug 11 15:18
    Borg-Can-Code review_requested #621
  • Aug 10 17:29
    Borg-Can-Code commented #620
  • Aug 10 16:15
    Borg-Can-Code commented #620
  • Aug 10 16:10
    Borg-Can-Code commented #620
  • Aug 10 16:09
    Borg-Can-Code commented #620
  • Aug 10 16:09
    Borg-Can-Code commented #620
  • Aug 10 16:08
    Borg-Can-Code commented #620
  • Aug 10 16:07
    Borg-Can-Code commented #620
  • Aug 10 15:41
    jeffwecan commented #620
  • Aug 10 15:35
    Borg-Can-Code opened #620
  • Aug 07 20:44
    Travis hvac/hvac (master) passed (1748)
  • Aug 06 15:32
    Travis hvac/hvac (develop) passed (1747)
  • Aug 05 18:53
    Travis hvac/hvac (develop) passed (1746)
  • Aug 05 18:40

    jeffwecan on develop

    Correct python-requests link U… Merge pull request #619 from go… (compare)

  • Aug 05 18:40
    jeffwecan closed #619
Caley
@caleyg
soo I have a double question there, we're just kiam to mask our interactions with the sts controller, basically it intercepts all calls to sts
let me invoke boto and get the access_key and secret_key
and it will give me an interesting error
access-key-from-boto3
secret-key-from-boto3
some-long-token-from-boto3
{'UserId': 'some-kiam-access-key:kiam-kiam', 'Account': '*****', 'Arn': 'arn:aws:sts::****:assumed-role/<domain>/kiam-kiam', 'ResponseMetadata': {'RequestId': 'some-request-id-uuid', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'some-request-id-uuid', 'content-type': 'text/xml', 'content-length': '460', 'date': 'Thu, 26 Mar 2020 19:21:15 GMT'}, 'RetryAttempts': 0}}
Traceback (most recent call last):
  File "docker_entrypoint.py", line 387, in <module>
    main()
  File "docker_entrypoint.py", line 378, in main
    client.auth.aws.iam_login(credentials.access_key, credentials.secret_key,role='some-role')
  File "/usr/local/lib/python3.7/dist-packages/hvac/api/auth_methods/aws.py", line 600, in iam_login
    json=params,
  File "/usr/local/lib/python3.7/dist-packages/hvac/adapters.py", line 174, in login
    response = self.post(url, **kwargs).json()
  File "/usr/local/lib/python3.7/dist-packages/hvac/adapters.py", line 103, in post
    return self.request('post', url, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/hvac/adapters.py", line 272, in request
    utils.raise_for_error(response.status_code, text, errors=errors)
  File "/usr/local/lib/python3.7/dist-packages/hvac/utils.py", line 32, in raise_for_error
    raise exceptions.InvalidRequest(message, errors=errors)
hvac.exceptions.InvalidRequest: error making upstream request: received error code 403 from STS: <ErrorResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <Error>
    <Type>Sender</Type>
    <Code>InvalidClientTokenId</Code>
    <Message>The security token included in the request is invalid.</Message>
  </Error>
  <RequestId>some-request-id-uuid</RequestId>
</ErrorResponse>
    session = boto3.Session()
    credentials = session.get_credentials()
    print(credentials.access_key)
    print(credentials.secret_key)
    print(credentials.token)
    print(session.client('sts').get_caller_identity())
    os.environ['REQUESTS_CA_BUNDLE'] = '/etc/ssl/certs/ca-certificates.crt'
    client = hvac.Client(url='some-vault')
    client.auth.aws.iam_login(credentials.access_key, credentials.secret_key,role='some-role')
the python that gets the above error is below the error ^
so I'm scratching my head here as to whats happening
I can auth with vault cli and get a token fine in the same environment its using the same sts and aws login method as hvac is trying to use
judging from the response from sts with the invalid token, it seems like there is another request happening to sts we're not seeing
Caley
@caleyg
and thank you so much for the almost immediate previous responses it means alot -- especially during these precarious times -- I hope you're in good health
Jeffrey Hogan
@jeffwecan
yeah I’m super lucky in that respect thankfully. Also its quite tuat
*that I happened to have just reinstalled gitter today after forgetting about it for some time :D
Caley
@caleyg
haha whew! thanks for both of those things!
Jeffrey Hogan
@jeffwecan
so presumably since you’re using STS, you’ll also need to get a session token from whatever you’re using to generate those credentials and pass it along to the hvac method call in this session_token parameter: https://github.com/hvac/hvac/blob/develop/hvac/api/auth_methods/aws.py#L568
most clients will expose that value in a AWS_SESSION_TOKEN env var / return value or some such
Caley
@caleyg
okay I'll allow sts GetSessionToken and see if that changes things
thanks so much
Caley
@caleyg
yeah none of that works... vault cli seems to be deriving information from the environment and sigv4 signing it, but for some reason hvac requires an access_key and secret_key and a token before trying to sigv4 signing it
its so weird that vault cli can do this but hvac just falls on itself
also AWS_SESSION_TOKEN isn't available in the k8 pod environment I'm in thats annotated with the kiam role arn so I can't access that, and get_session_token() is unavailable to me since my creds are temporary creds issued from the kiam masked service (https://github.com/uswitch/kiam)
Jeffrey Hogan
@jeffwecan
ah sigv4 eh? that reminds me of something,… lemme look around right quick
Caley
@caleyg
many thanks
Jeffrey Hogan
@jeffwecan
so the thing I was thinking of is this old style (in the context of hvac) method which handles creating a sigv4 signature: https://github.com/hvac/hvac/blob/develop/hvac/v1/__init__.py#L603
but I see that the iam_login() method you’ve been trying includes the same logic
so I don’t have anything else to offer at the moment without fiddling with kiam and trying to replicate myself, sorry :(
Caley
@caleyg
all the thanks for your input so far -- if you wanna fiddle with kiam and get back to me that would be awesome :D
I really dont wanna install vault cli on the container and use that
Jeffrey Hogan
@jeffwecan
:+1: we’ll see if I manage to find the spare time or not, but if I do I’ll be sure to hit ya up haha
Caley
@caleyg
thanks!
Caley
@caleyg

I went ahead so I'm not blocking the business I am forgoing hvac for now, and subprocessing out to vault cli to do the following

    os.environ['VAULT_ADDR'] = 'https://some.dev.vault.net'
    vault_cmd = 'vault login -method=aws -token-only role=some-role'
    vault_token = subprocess.run(vault_cmd, shell=True, universal_newlines=True, check=True)
    print(vault_token)

which yields a token which as been redacted below:

INTENTIONALLY REDACTED
CompletedProcess(args='vault login -method=aws -token-only role=some-role', returncode=0)

not exactly what we wanted to do but its all I got for some reason vault cli doesn't mind being behind kiam (https://github.com/uswitch/kiam) and using the sts creds from that service

Jeffrey Hogan
@jeffwecan
hmm interesting. whats that AWS vault role’s type?
i.e., is it indeed iam or ec2?
Caley
@caleyg
its iam for sure
Jeffrey Hogan
@jeffwecan
word. out of curiosity, whats the vault server and client versions you’re using there?
Caley
@caleyg
$ terraform state show vault_aws_auth_backend_role.service_ro\[\"some-role\"\]
# vault_aws_auth_backend_role.service_ro["some-role"]:
resource "vault_aws_auth_backend_role" "service_ro" {
    allow_instance_migration        = false
    auth_type                       = "iam"
    backend                         = "aws"
   bound_account_ids               = []
    bound_ami_ids                   = []
    bound_ec2_instance_ids          = []
    bound_iam_instance_profile_arns = []
    bound_iam_principal_arns        = [
        "arn:aws:iam::REDACTED:role/some/role/name",
    ]
?
Jeffrey Hogan
@jeffwecan

also one other random thought, what region(s) are your clients in?

(thinking of something like https://github.com/hashicorp/vault-ruby/pull/161#issuecomment-355723269)

Caley
@caleyg
vault server version Version 0.10.4
vault client version Vault v1.2.3
its a little old granted, its on our backlog to upgrade :P
also us-east-1
Jeffrey Hogan
@jeffwecan
k so at least the region bit is probably all good
Caley
@caleyg
let me know if I get you anymore information
Jeffrey Hogan
@jeffwecan
will do!
Caley
@caleyg
we thought it might be related to bound_account_ids but we tried that and nothing really changed
Jeffrey Hogan
@jeffwecan
though at this point, it would be super helpful if you want to copy/paste some of this info into an issue on the hvac repo so I don’t lose track of it :)
Caley
@caleyg
totally
I'll get on that in a bit
Caley
@caleyg
hvac/hvac#564 added, if I'm missing any details please let me know
Jeffrey Hogan
@jeffwecan
thanks so much!
John Maguire
@jm96441n
hey all! I was thinking of opening a PR for a contribution to the hvac lib and wanted to get some extra opinions from the maintainers on the potential change prior to just opening the PR
so it would be around the Client.renew_token method, it seems a bit unclear at first that based on the param passed in that the method would hit differing endpoints, I was thinking about separating out a separate Client.renew_self_token or something along those lines that would cover hitting the renew-self endpoint
we just ran into an issue where we were passing in the token= param assuming that if the client had access to renew it's own token that it would be able to renew it when passing the token in, but it doesn't seem that's the case as the client would need to have access to the renew endpoint which could give services more access than we would want them to have (hopefully that all made sense)