let me invoke boto and get the access_key and secret_key
and it will give me an interesting error
access-key-from-boto3
secret-key-from-boto3
some-long-token-from-boto3
{'UserId': 'some-kiam-access-key:kiam-kiam', 'Account': '*****', 'Arn': 'arn:aws:sts::****:assumed-role/<domain>/kiam-kiam', 'ResponseMetadata': {'RequestId': 'some-request-id-uuid', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'some-request-id-uuid', 'content-type': 'text/xml', 'content-length': '460', 'date': 'Thu, 26 Mar 2020 19:21:15 GMT'}, 'RetryAttempts': 0}}
Traceback (most recent call last):
File "docker_entrypoint.py", line 387, in <module>
main()
File "docker_entrypoint.py", line 378, in main
client.auth.aws.iam_login(credentials.access_key, credentials.secret_key,role='some-role')
File "/usr/local/lib/python3.7/dist-packages/hvac/api/auth_methods/aws.py", line 600, in iam_login
json=params,
File "/usr/local/lib/python3.7/dist-packages/hvac/adapters.py", line 174, in login
response = self.post(url, **kwargs).json()
File "/usr/local/lib/python3.7/dist-packages/hvac/adapters.py", line 103, in post
return self.request('post', url, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/hvac/adapters.py", line 272, in request
utils.raise_for_error(response.status_code, text, errors=errors)
File "/usr/local/lib/python3.7/dist-packages/hvac/utils.py", line 32, in raise_for_error
raise exceptions.InvalidRequest(message, errors=errors)
hvac.exceptions.InvalidRequest: error making upstream request: received error code 403 from STS: <ErrorResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
<Error>
<Type>Sender</Type>
<Code>InvalidClientTokenId</Code>
<Message>The security token included in the request is invalid.</Message>
</Error>
<RequestId>some-request-id-uuid</RequestId>
</ErrorResponse> session = boto3.Session()
credentials = session.get_credentials()
print(credentials.access_key)
print(credentials.secret_key)
print(credentials.token)
print(session.client('sts').get_caller_identity())
os.environ['REQUESTS_CA_BUNDLE'] = '/etc/ssl/certs/ca-certificates.crt'
client = hvac.Client(url='some-vault')
client.auth.aws.iam_login(credentials.access_key, credentials.secret_key,role='some-role')
the python that gets the above error is below the error ^
so I'm scratching my head here as to whats happening
I can auth with vault cli and get a token fine in the same environment its using the same sts and aws login method as hvac is trying to use
judging from the response from sts with the invalid token, it seems like there is another request happening to sts we're not seeing
judging from the response from sts with the invalid token, it seems like there is another request happening to sts we're not seeing
*that I happened to have just reinstalled gitter today after forgetting about it for some time :D
so presumably since you’re using STS, you’ll also need to get a session token from whatever you’re using to generate those credentials and pass it along to the hvac method call in this
session_token parameter: https://github.com/hvac/hvac/blob/develop/hvac/api/auth_methods/aws.py#L568
most clients will expose that value in a
AWS_SESSION_TOKEN env var / return value or some such
thanks so much
its so weird that vault cli can do this but hvac just falls on itself
also
AWS_SESSION_TOKEN isn't available in the k8 pod environment I'm in thats annotated with the kiam role arn so I can't access that, and get_session_token() is unavailable to me since my creds are temporary creds issued from the kiam masked service (https://github.com/uswitch/kiam)
so the thing I was thinking of is this old style (in the context of hvac) method which handles creating a sigv4 signature: https://github.com/hvac/hvac/blob/develop/hvac/v1/__init__.py#L603
but I see that the
iam_login() method you’ve been trying includes the same logic
so I don’t have anything else to offer at the moment without fiddling with kiam and trying to replicate myself, sorry :(
I really dont wanna install vault cli on the container and use that
I went ahead so I'm not blocking the business I am forgoing hvac for now, and subprocessing out to vault cli to do the following
os.environ['VAULT_ADDR'] = 'https://some.dev.vault.net'
vault_cmd = 'vault login -method=aws -token-only role=some-role'
vault_token = subprocess.run(vault_cmd, shell=True, universal_newlines=True, check=True)
print(vault_token)
which yields a token which as been redacted below:
INTENTIONALLY REDACTED
CompletedProcess(args='vault login -method=aws -token-only role=some-role', returncode=0)not exactly what we wanted to do but its all I got for some reason vault cli doesn't mind being behind kiam (https://github.com/uswitch/kiam) and using the sts creds from that service
i.e., is it indeed
iam or ec2?
$ terraform state show vault_aws_auth_backend_role.service_ro\[\"some-role\"\]
# vault_aws_auth_backend_role.service_ro["some-role"]:
resource "vault_aws_auth_backend_role" "service_ro" {
allow_instance_migration = false
auth_type = "iam"
backend = "aws"
bound_account_ids = []
bound_ami_ids = []
bound_ec2_instance_ids = []
bound_iam_instance_profile_arns = []
bound_iam_principal_arns = [
"arn:aws:iam::REDACTED:role/some/role/name",
]also one other random thought, what region(s) are your clients in?
(thinking of something like https://github.com/hashicorp/vault-ruby/pull/161#issuecomment-355723269)
its a little old granted, its on our backlog to upgrade :P
also us-east-1
I'll get on that in a bit
so it would be around the
Client.renew_token method, it seems a bit unclear at first that based on the param passed in that the method would hit differing endpoints, I was thinking about separating out a separate Client.renew_self_token or something along those lines that would cover hitting the renew-self endpoint
we just ran into an issue where we were passing in the
token= param assuming that if the client had access to renew it's own token that it would be able to renew it when passing the token in, but it doesn't seem that's the case as the client would need to have access to the renew endpoint which could give services more access than we would want them to have (hopefully that all made sense)
Client.renew_token docs for reference: https://hvac.readthedocs.io/en/stable/_modules/hvac/v1.html#Client.renew_token