Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 31 2019 14:07
    jugalde-r7 commented #2352
  • Jan 31 2019 09:36
    takeyourhatoff commented #2427
  • Jan 31 2019 09:35
    takeyourhatoff commented #2427
  • Jan 30 2019 23:10
    diehlaws commented #2427
  • Jan 30 2019 23:09
    diehlaws labeled #2427
  • Jan 30 2019 23:09
    diehlaws labeled #2427
  • Jan 30 2019 21:47
    diehlaws labeled #2352
  • Jan 30 2019 21:47
    diehlaws commented #2352
  • Jan 30 2019 21:12
    diehlaws commented #2342
  • Jan 30 2019 20:58
    diehlaws assigned #2427
  • Jan 30 2019 20:54
    diehlaws unlabeled #81
  • Jan 30 2019 20:54
    diehlaws unlabeled #142
  • Jan 30 2019 20:54
    diehlaws unlabeled #618
  • Jan 30 2019 20:54
    diehlaws unlabeled #81
  • Jan 30 2019 20:54
    diehlaws unlabeled #142
  • Jan 30 2019 20:54
    diehlaws unlabeled #619
  • Jan 30 2019 20:54
    diehlaws unlabeled #628
  • Jan 30 2019 20:54
    diehlaws unlabeled #568
  • Jan 30 2019 20:54
    diehlaws unlabeled #521
  • Jan 30 2019 20:54
    diehlaws unlabeled #487
Simon Woldemichael
@swoldemi
@mrichman How are you reading the types of the keys from the table?
Mark Richman
@mrichman
func discoverSchema(tableName string) error {
    svc = dynamodb.New(getSession())
    response, err := svc.DescribeTable(&dynamodb.DescribeTableInput{
        TableName: aws.String(tableName),
    })

    if err != nil {
        return err
    }

    // get pk and sk attrib names
    for attrib := range response.Table.KeySchema {
        if *response.Table.KeySchema[attrib].KeyType == "HASH" {
            pkName = *response.Table.KeySchema[attrib].AttributeName
        }
        if *response.Table.KeySchema[attrib].KeyType == "RANGE" {
            skName = *response.Table.KeySchema[attrib].AttributeName
        }
    }

    // get pk and sk attrib types:
    // S - the attribute is of type String
    // N - the attribute is of type Number
    // B - the attribute is of type Binary
    for attrib := range response.Table.AttributeDefinitions {
        if *response.Table.AttributeDefinitions[attrib].AttributeName == pkName {
            pkType = *response.Table.AttributeDefinitions[attrib].AttributeType
        }
        if *response.Table.AttributeDefinitions[attrib].AttributeName == skName {
            skType = *response.Table.AttributeDefinitions[attrib].AttributeType
        }
    }
    return nil
}
Simon Woldemichael
@swoldemi
@mrichman Passing a []byte should be fine, since the SDK will encode it for you to probably format the PutItem request. I get what you mean on not being able to easily use Marshal. What you currently have should work
Mark Richman
@mrichman
Thanks @swoldemi I'll report back if I have any issues with that. In the meantime, I do have one other issue. After my load test app runs for a few minutes, I hit this error: RequestError: send request failed\ncaused by: Post https://dynamodb.us-east-1.amazonaws.com/: dial tcp: lookup dynamodb.us-east-1.amazonaws.com on 172.31.0.2:53: no such host. I'm thinking this is a linux kernel problem (ulimit?) and not the SDK itself. I'm running on a t3.medium instance, if that helps.
Mark Richman
@mrichman
LOL the illustrious Dave Cheney chimed in https://github.com/golang/go/issues/18588#issuecomment-661807513
Simon Woldemichael
@swoldemi
@mrichman Lol nice. Too many open files does look like a ulimit error. Also worth checking if your EC2 instance allows access to the outside internet. Are you using a dynamodb VPC endpoint?
Tharaka Kumburuthenne
@danakum
hi i would like to know how use username password in aws
hi i would like to know how use username password in aws
as authentication in SES
Simon Woldemichael
@swoldemi
@danakum Can you elaborate on what you mean? Are you wanting to use SES to send emails with an SMTP username and password?
Tharaka Kumburuthenne
@danakum
yes @swoldemi
Simon Woldemichael
@swoldemi
@danakum You should be able to use the native smtp package: https://golang.org/pkg/net/smtp/#SendMail
There are a few examples in other languages here: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/send-using-smtp-programmatically.html
For the second parameter of SendMail, you should be able to just use a PlainAuth and pass your username and password there: https://golang.org/pkg/net/smtp/#PlainAuth
Nicolas Sitbon
@nsitbon
Hi I'm using the Upload method (https://docs.aws.amazon.com/sdk-for-go/api/service/s3/s3manager/#Uploader.Upload) to push a file in a S3 bucket. The method returns a https://docs.aws.amazon.com/sdk-for-go/api/service/s3/s3manager/#UploadOutput containing an UploadId. How can I pass it back to the same Upload method to resume the upload of big files when an error occurred? Best regards
Simon Woldemichael
@swoldemi
@nsitbon Upload won't return an UploadId if there's an error uploading a file that is too big for 1 part. You'll just get an error that says something like "upload multipart failed". Best way might be to run the multipart upload yourself and resume the upload when an error occurs on a specific part.
Simon Woldemichael
@swoldemi
@nsitbon Actually I'm wrong, the MultiUploadError returns has an UploadId() too, not just the message. If you set, LeavePartsOnError to true on your Uploader, you can resume from where it failed. I think there are 2 S3 APIs that will be helpful, ListMultiPartUploads and ListParts.
To see where it failed you can call s3.ListMultiPartUploads to get the process, just need to include the UploadId: https://docs.aws.amazon.com/sdk-for-go/api/service/s3/#ListMultipartUploadsInput
It might be better to call s3.ListParts instead to see where to resume from, since the list of Parts that are returned give you the part number and the byte where that part ended, but I'm not sure since I haven't done this before: https://docs.aws.amazon.com/sdk-for-go/api/service/s3/#S3.ListParts
From that info you should be able to figure out where to resume, with a bit of math. It doesn't look like there's a way to do this with just s3manager.Uploader.
Aditya Kumar Gupta
@adityakrgupta25
I have been trying to run WaitUntilTasksStopped after triggering a task on the ECS, but when the tasks fail with non zero exit code, it does not return an error. I have proded around a bit and tried to look at the returned response from aws when the tasks returns. Turns out the failiure field is empty, but i can see is exit code is non zero. What can be done if non-zero exit codes are not being captured as error by WaitUntilTasksStopped by the SDK?
Simon Woldemichael
@swoldemi
@adityakrgupta25 What behavior are you trying to produce other than waiting for the task to stop? Maybe calling DescribeTasks then reading the exit code from there?
puvvadav
@puvvadav
Is there any way to specify client timeout ? For example if COS is not responding client seems to be stuck forever while downloading an object.
Simon Woldemichael
@swoldemi
@puvvadav You can set a timeout on the context you pass to GetObjectWithContext: https://docs.aws.amazon.com/sdk-for-go/api/service/s3/#S3.GetObjectWithContext
You can also provide a custom HTTP client when creating your session, but I don't think that will timeout while reading the response body so you'll need to provide a context: https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/custom-http.html
Simon Woldemichael
@swoldemi
Likewise on DownloadWithContext if you're using s3manager's downloader: https://docs.aws.amazon.com/sdk-for-go/api/service/s3/s3manager/#Downloader.DownloadWithContext
VictorHSB
@victorhsb
hello, good evening! does anyone here know if i can request ona specific taks inside a service on ECS? i know that with ECS's service discovery i can call by "service.cluster" but can i do something like this but specifying which instance i want to call? i'm trying to make multiple instances talk to each other. (talking specifically about the sdk-go, i know i can list the instances of a service and get the instance id's with it)
1 reply
Josh
@jnprogrammer
Hello everyone
Promaethius
@Promaethius
This may be a silly question but I don't have access to an IC C2S environment to test it out. Does the aws-go-sdk automatically resolve services to FIPS compliant endpoints? This issue says no
aws/aws-sdk-go#2219
but I see FIPS resolution for us-gov--1 in endpoints.json
https://github.com/aws/aws-sdk-go/blob/master/models/endpoints/endpoints.json
Without any modification or custom resolver functions, will aws-go-sdk work with a given us-gov-
-1 region out of the box and resolve to available FIPS endpoints?
1 reply
Miles Conn
@MilesConn
Hi I'm fetching personal information about users from cognito. I have a list of usernames but I'm trying to come up with a fast way to query cognito. Seems like ListUsers only allows a single string to filter. Any ideas? Right now getting personal information is super slow. I'm not quite ready to move personal information to my own DB
1 reply
Michael Bonner
@bonner
Hello, anyone if/how to write data to s3 in parquet? possible out of the box with the go sdk?
4 replies
Charlie
@zhouchunli
session name is defined by ourself?
2 replies
Aditya Kumar Gupta
@adityakrgupta25
How to check if auto-scaling is enabled for an ECS service?
4 replies
Aditya Kumar Gupta
@adityakrgupta25
ecs'sWaitUntilServicesStable() takes as input a list of services and returns just the error. If there's an error, how can I know which Services were the ones that did not become stable?
1 reply
Eric Bressler
@ebressler
has anyone seen xmlutil.XMLToStruct taking a lot of memory? It seems to be holding on to a lot and thus putting my process to 1gb or so of used memory. Seems to be using at least half of the allocated amount.
7 replies
this is mostly from running ListObjectsV2 calls
Eric Bressler
@ebressler
hmm, I guess nobody has any great guidance of what to look for here. it seems like I should be closing something out that I am not. I am going to have to narrow down what is holding references I guess.
Michael Lindes
@mlindes
anyone know? - using the aws-sdk-go, is it possible to do an sts-assume-role with environment variables only? ie - not using the ~/.aws/credentials file?
2 replies
Kostas Papageorgiou
@kostaspap

I have a Lambda application that assumes a role in a different AWS Account using STS temporary credentials and reads some data from S3.
The credentials have expiration time of 1 hr.
I am using the creds provider returned from stscreds.NewCredentials . I am setting the ExpiryWindow of the provider to 1 minute.

From time to time (once every day) I will see an "Access Denied" exception when trying to get an object from S3. The call will succeed when my Lambda retries (after few minutes)

Anyone has seen that? Is it possible that this is due to some AWS glitch and I just have retry such an error programmatically? Or is possible that the 1minute ExpiryWindow that I have configured is somehow very low?

Parshuram Patil
@parshuram-patil
what is the import statement for batch service in golang?
2 replies
Иван Сердюк
@oceanfish81_twitter
Could some tell why I am getting this:
aws-sdk-go/aws$ go test ./...
credentials/plugincreds/provider.go:96:2: package plugin is not in GOROOT (/home/oceanfish81/gollvm_dist/src/plugin)
Иван Сердюк
@oceanfish81_twitter
?
5 replies
Ivan Tham
@pickfire
How come after I upgrade aws version I got some errors running test?
pkg/cloud/aws/autoscalinggroup_test.go:30:29: cannot use client (type *mockaws.MockAutoScalingAPI) as type autoscalingiface.AutoScalingAPI in field value:
        *mockaws.MockAutoScalingAPI does not implement autoscalingiface.AutoScalingAPI (missing CancelInstanceRefresh method)
3 replies
shreyas
@techiespace
I'm trying to delete all files in a folder using go sdk. can someone please point me to a example?
1 reply
nehaguleria-softuvo
@nehaguleria-softuvo
HI
misschak
@misschak
Having problem with DeleteObject on object key with '@' character
4 replies
FHIDMAN FALTZ
@fhidmanf
he guys, how I check if the status value published to the Amazon SNS topic is SUCCEEDED, I wanna use the textract api
2 replies
Aditya Kumar Gupta
@adityakrgupta25
How to set ConditionExpression for dynamodb.UpdateItemInput using dynamodbiface.expression
https://stackoverflow.com/questions/64502401/how-to-set-conditionexpression-for-dynamodb-updateiteminput-using-dynamodbi
I need help with the above questions
Mike Dalrymple
@mousedownmike
@adityakrgupta25 I just posted an answer for you.
Severyn Lisovskyi
@sev3ryn
hi, I'm trying to use s3manager.BatchDeleteIterator with custom iterator - that will skip few of the keys but I can't figure out how to access s3 filenames from DeleteListIterator
    keysToSkip := map[string]struct{}{
        "version": {},
        "info":    {},
    }
    fn := func(dli *s3manager.DeleteListIterator) {
        // TODO add check
    }
    s3manager.NewDeleteListIterator(s3.New(sess), input, fn)
2 replies
AnnikaLarissa
@AnnikaLarissa
Hey, I got a really specific question and I hope someone can help me out. I am using s3 (sdk for go) and s3 does not have a hierarchy with folders and subfolder. So I named my files foo/bar/1/etc. Using prefix gives me ALL the files with that prefix, but I only want to go one step down in hierarchy. I have been reading the API and looking around on forums, but I cannot find a solution for this problem in golang. I did find: boto/boto3#134. This solution should work for go as well, as you can set prefixes and delimiters. But I have no idea how this CommonPrefix works in golang.
1 reply
Gowrisankar
@gowrisankar22
hi
are there any golang samples example available for aws privatelink to connect with rds postgres ?
Nayeem Junaid
@NayeemJunaid
I cant find Golang - ElastiCache example code at all. Please help!
CSP
@CSP197

Hi, is there a way to get a list of profile names in the shared name profiles file ~/.aws/credentials

For aws-cli, I run aws configure list-profiles, but what would be the golang sdk equivalent?