Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
    Jarrett Vance
    I need to process a CSV file from S3 and convert it to a JSON file and put back into S3. The files could be larger than 16MB so I want to stream this data without saving to disk or holding it all in memory. I plan to do the work with a Lambda. My trouble is the S3 client (and TransferUtility) relies on HashStream which requires prior knowledge of content length. I won't know the length ahead of time. Any suggestions?
    Jarrett Vance
    My stream can't be seekable because I don't want to keep the whole thing in memory. I've tried wrapping my stream in both PartialReadOnlyWrapperStream and ReadOnlyWrapperStream
    Jarrett Vance
    {System.Net.Http.WinHttpException (0x80070057): The parameter is incorrect}
    Jarrett Vance
    working on a solution using multpart upload. however, if my sizes are all greater than 5MB I'm still getting an error Amazon.S3.AmazonS3Exception: Your proposed upload is smaller than the minimum allowed size
    Jarrett Vance
    Figured out that with PutObjectRequest, it auto-resets the stream but UploadPartRequest does not
    Jarrett Vance
    I got this working! Please improve TransferUtility to support streams of unknown length so you don't have to drop to the low level multipart API :thanks:
    Jarrett Vance
    I opened an issue to encourage discussion around .NET 2.1 improvements: aws/aws-sdk-net#971

    Any tips on how to make lambda handler c# code more efficient from .NET perspective?
    Some of the example questions I am trying to address:
    1) WIll making lambda functions static increase reuse of Lambda context and improve performance?
    2) If I make Functions class ( which has all lambda handlers) singleton class, will it improve performance?
    3) If I make constant/readonly variables and share it across lambda functions, will it improve performance?

    If anyone has any information, please suggest

    Taras Romanyk
    1) I don't think that static variables are good way to keep any information. Container of Lambda function could be restarted and you lose your data. AFAIK it must be restarted each 4 hours. Google "cold start" for more information.
    Try to use key-value storage (like Redis) to persist shared information. But in this case it will not be REST architecture (REST should not keep a client state on the server, be stateless).
    2) It doesn't look as good idea also. At least, you will have to think about locking and state sharing between calls in this case. Make a test and write your results. It is interesting.
    3) I guess - no. Changes will be unseen and doesn't worth efforts. But I am doing like for simplifying development process.
    Hi, I would like to build aws sdk for Unity but it shouldn't use the google class GoogleCloudMessaging (AWSUnityGCMWrapper), I only want it to use S3 service without authentication. How could I have this?
    Jez Reel Maghuyop

    Hi guys, I'm new to AWS and I've been trying to fix an issue I have with the appsettings.json... been looking for a fix for 3days now ..

    I just wanted to know if there is a way to specify which Appsettings to use when deploying a .Net Core application? I have (Dev, Staging and Production) I'm using the Publish to Elastic Beanstalk in my Visual Studio 2017.

    Jez Reel Maghuyop
    anyone deployed IdentityServer4 on AWS?
    Stdrovia Blackburn
    @W2K Did you ever resolve that invalid token exception?
    Kevin Amditis
    having a problem trying to implement the new CognitoAuthentication Extension in my Standalone Unity Game. If i could get someones attention who is familiar with the subject I would greatly appreciate it. This has been very frustrating over the last week
    I am attempting to follow the "authenticate with SRP" flow for AWS Cognito user pool authorization for .NET in my Unity 2018.2.0 project. I have all the AWS resources in my project and get no errors in VS2017 or Unity. Everything up to the call for await StartWithSrpAuthAsync() works fine. Calling this function however results in no exceptions, results, or errors and never makes it to the next line of code. I cannot find anyone else experiencing this issue. This seems exclusive to Unity and the newer CognitoAuthenticationExtension Library for .Net. The code I'm using was found under "Authenticating with Secure Remote Protocol (SRP)" from Amazon documentation found at: https://aws.amazon.com/blogs/developer/cognitoauthentication-extension-library-developer-preview/
    Are there any plans on adding the CognitoAuthentication extension to the Unity packages?
    I created an asp.net core application that uses Cognito and authenticates with Google. The authentication works very well. As a next step I want to use the logged-in user's credentials to access s3 bucket. I have created an identity pool with cognito in it, but I am not sure how to proceed further. I could not find any examples either. Could you please help? Everywhere I look, it uses username/password and not any of the social identity providers.
    Niraj Bhatt
    getting exeption "mismatch between session identity id and request identity id" during Congito Sync Synchronize method when trying to login multiple users on same device. If anyone can help, I would be very greatful.
    Niraj Bhatt
    solved above issue by making sure to create new CognitoAWSCredentials instance for each CognitoSyncManager
    Niraj Bhatt
    I am facing more or less similar issue for unauthenticated call.
    It seems during multiple login user, it tries to access previous user identity data that was cached. So I think I need to clear credentials
    I have found 3 methods of CognitoAWSCredentials to clear cache:
    Only Clear seems to work. However I have read documents and it mentions that we should use ClearCredentials or ClearIdentityCache.
    Please let me know if anyone has faced this issue
    Sachin Dole
    Hello, I have a question. I need to sign requests using SigV4 in a Lambda function, on my workstation and in CodeBuild. The mechanism to obtain credentials varies a bit in each environment. My question is, how can I get access to credentials without writing environment aware code. In other words, is there any method or property(ies) in the SDK that will give the access key, secret key and session token? Thanks! I appreciate it.
    Hi guys, I'm trying to integrate AWS API to my asp.net mvc4 project and I'm not sure what type of credential I should use. Manually i log-in to AWS using the third-party app (okta), so I don't have credentials directly from AWS. Can I get\generate access key or smth else for login throught API ? Or I should ask my AWS admin anyway to provide some creds ?
    hello, may I ask whether Visual Studio Community for Mac version supports AWS Toolkit? If yes, how to install it? Thanks.

    I am developing an app with UNITY 2017.2.3f1 and the aws-sdk-unity_3.3.396.2
    S3 works fine which means that many elements are OK. The code works fine for MAC and Android
    However when I try to post an element on DynamoDB with the following code

    var rec = new Document();
    rec["id"] = 1;//recName;
    rec["Lat"] = 2;//Input.location.lastData.latitude;
    rec["Long"] = 3;//Input.location.lastData.longitude;
    rec["Time"] = 4;//recTime;

                table.PutItemAsync(rec, (r) => {
                    Debug.Log("\n rec has been poted " + r.Exception + " *222444*****************");
                    textnotification.text = "Archivo subido con éxito";
                    Invoke("cleanText", 3);
                    butSender.interactable = false;
                    butPlayer.interactable = true;
                    butRecorder.interactable = true;
                    hasnotbeensent = false;

    it complains with the following error.

    System.InvalidOperationException: No converter configured for type System.String
    at Amazon.DynamoDBv2.ConverterCache.GetConverter (System.Type type) [0x00000] in <filename unknown>:0
    at Amazon.DynamoDBv2.DynamoDBEntryConversion.ConvertToEntry (System.Type inputType, System.Object value) [0x00000] in <filename unknown>:0
    at Amazon.DynamoDBv2.DocumentModel.UnconvertedDynamoDBEntry.ConvertToAttributeValue (Amazon.DynamoDBv2.DocumentModel.AttributeConversionConfig conversionConfig) [0x00000] in <filename unknown>:0
    at Amazon.DynamoDBv2.DocumentModel.Document.ToAttributeMap (Amazon.DynamoDBv2.DynamoDBEntryConversion conversion, IEnumerable1 epochAttributes) [0x00000] in <filename unknown>:0 at Amazon.DynamoDBv2.DocumentModel.Table.PutItemHelper (Amazon.DynamoDBv2.DocumentModel.Document doc, Amazon.DynamoDBv2.DocumentModel.PutItemOperationConfig config) [0x00000] in <filename unknown>:0 at Amazon.DynamoDBv2.DynamoDBAsyncExecutor+<>c__DisplayClass1_01[T].<ExecuteAsync>b__0 (System.Object state) [0x00000] in <filename unknown>:0 * *

    I think it is a Bug but I am not sure.

    Hippie Techie

    Hello fellow devs,

    Am facing a problem with the delimiter parameter of the request object for listing objects in the s3 bucket.

    it returns NOT IN results instead of in.

    Anyone faced this problem before can lead me to another possible direction to solve this?

    Hippie Techie
    Jan Chvojka

    Hi all, I have one question and I can't find answer for it (SO, google, numerous tries my own code). How can I instantiate AmazonCognitoIdentityProviderClient?
    I'm using .net core 2 and the code is really simple:

    var credentials = new EnvironmentVariablesAWSCredentials();
    CognitoProvider = new AmazonCognitoIdentityProviderClient(credentials, RegionEndpoint.EUCentral1);

    But AmazonCognitoIdentityProviderClient throws exception
    System.IO.FileNotFoundException: 'Could not find file 'C:\Users\USER\.aws\credentials'.'
    I know what it means, but why is it throwing that exception? I want to use environment credentials and they are all right (EnvironmentVariablesAWSCredentials doesnt complain and Environment.GetEnvironmentVariable on AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY returns correct values).
    I can provide the file on my local environment but I can't do that on production. How am I suppose to pass credentials to cognito client? Am I missing something?

    Can new AmazonS3Client(); use configurations from appsettings.json ?
    I have in my appsettings.json: "AWS": { "Region": ..., "AccessKeyId": ..., "SecretAccessKey":... }, but I cannot create a s3 client
    nvm...this is not a feature
    Hi All, I'm trying to work with Glacier to files from a vault with thousands of files.. not ideal, but that's the job..
    I'm using the ArchiveTransferManager with DownloadAsync
    and as you can imagine 4+ hours per file is too slow.
    Is there a way to initiate the DownloadAsync method using the Expedited Tier?
    Omer Katz
    Hi guys
    I'm trying to list all the tables of a dynamodb instance using the sdk but I'm getting 'AmazonDynamoDBClient.ListTables()' is inaccessible due to its protection level
    This directly contradicts the documentation
    I'm using dot net core 2.1
    On AWS lambda
    Norm Johanson
    @thedrow For .NET Core you need to use the async version of the API because that is all the underlying HttpClient for .NET Core supports.
    @jaeckyl Can you open a GitHub issue for the expedited tier so we can add the feature request to our backlog. The ArchiveTransferManager high level abstraction hasn't been updated for that Glacier feature
    Hi guys, i am trying to do a QueryAsync in DynamoDb with a bool filter. I am getting no result back from the sdk. Are there some problem with filters on bool values?
    For other values it works
           opConfig.QueryFilter =
                new List<ScanCondition>
                    new ScanCondition("Status", ScanOperator.Equal, 10),
                    new ScanCondition("Status", ScanOperator.Equal, 4),
                    new ScanCondition("PlanType", ScanOperator.Equal, "SML"),
                    //new ScanCondition("NormalRoadConditions", ScanOperator.Equal, true)
    The last one gives 0 result count even thou there should be results
            List<string> partionKeys = new List<string>() { "52049E96-14FA-4960-B3CE-F7400C4AC6CD", "01F3DF08-4D0F-4E92-BBE3-8E46BA77E73B", "52117BC1-7256-4C36-868E-465DB2178286" };
            if (!string.IsNullOrEmpty(TableName))
                AWSConfigsDynamoDB.Context.TypeMappings[typeof(Plan)] = new Amazon.Util.TypeMapping(typeof(Plan), TableName);
            var start = DateTime.Now;
            int totalCount = 0;
            var config = new DynamoDBContextConfig { Conversion = DynamoDBEntryConversion.V2 };
            var client = new AmazonDynamoDBClient(new StoredProfileAWSCredentials("sandbox"), RegionEndpoint.EUWest1);
            DbContext = new DynamoDBContext(client, config);
            var opConfig = new DynamoDBOperationConfig { IndexName = "OrganizationId-Created-index" };
            opConfig.QueryFilter =
                new List<ScanCondition>
                    new ScanCondition("Status", ScanOperator.Equal, 10),
                    new ScanCondition("Status", ScanOperator.Equal, 4),
                    new ScanCondition("PlanType", ScanOperator.Equal, "SML"),
                    //new ScanCondition("NormalRoadConditions", ScanOperator.Equal, true)
            foreach (var partionKey in partionKeys)
                var result = Program.DbContext.QueryAsync<Plan>(partionKey.ToLower(), opConfig);
                while (!result.IsDone)
                    var plans = result.GetNextSetAsync().Result;
                    totalCount += plans.Count;
    Here is the whole code