Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Aug 16 18:17

    aws-sdk-php-automation on 3.233.3

    (compare)

  • Aug 16 18:17

    aws-sdk-php-automation on master

    Update models for release 3.233.3 release (compare)

  • Aug 16 17:56
    ziyanli-amazon synchronize #2507
  • Aug 15 18:19

    aws-sdk-php-automation on 3.233.2

    (compare)

  • Aug 15 18:19

    aws-sdk-php-automation on master

    Update models for release 3.233.2 release (compare)

  • Aug 12 21:27
    ziyanli-amazon synchronize #2507
  • Aug 12 18:21

    aws-sdk-php-automation on 3.233.1

    (compare)

  • Aug 12 18:21

    aws-sdk-php-automation on master

    Update models for release 3.233.1 release (compare)

  • Aug 12 15:13
    yenfryherrerafeliz labeled #2508
  • Aug 12 15:13
    yenfryherrerafeliz unlabeled #2508
  • Aug 12 15:12
    yenfryherrerafeliz commented #2508
  • Aug 12 14:21
    yenfryherrerafeliz assigned #2508
  • Aug 12 03:27
    jaimeburnap edited #2508
  • Aug 12 03:26
    jaimeburnap edited #2508
  • Aug 12 03:26
    jaimeburnap edited #2508
  • Aug 12 03:25
    jaimeburnap edited #2508
  • Aug 12 03:25
    jaimeburnap opened #2508
  • Aug 12 03:25
    jaimeburnap labeled #2508
  • Aug 12 03:25
    jaimeburnap labeled #2508
  • Aug 12 00:27
    georgewoofbates commented #2302
Patrik Patie Gmitter
@patie
hi, im trying to do synchronous command pool, but i give Catchable Fatal Error: Argument 1 passed to Aws\Handler\GuzzleV5\GuzzleHandler::Aws\Handler\GuzzleV5{closure}() must be an instance of Exception, string given, called in ......../vendor/guzzlehttp/promises/src/Promise.php on line 199 and defined
any idea ? :)
i using guzzle 5.3 which is supported, when im looking at packagist
Patrik Patie Gmitter
@patie
ok with guzzle 6.1 there is no problem.
Zxurian
@Zxurian
when using the uploadAsync() method, should it start uploading as soon as the method is called, or is there somthing additional I have to call for it to start uploading?
Jonathan Eskew
@jeskew
@Zxurian It should start as soon as the method is called
It returns a promise, so you can call ->wait() on that if you need to block until the upload is complete
Zxurian
@Zxurian
that's what I thought as well, however when I was issuing the ->uploadAsync() method, nothing was getting uploaded. I had a separate timer instead of calling wait, but checked on the value returned by the promise ->then() method, and also checking S3 directly, the file never uploaded
Jonathan Eskew
@jeskew
I’m going to run a couple tests on that
Zxurian
@Zxurian
@jeskew I'll see if I can strip my code down to a simple version for testing
Jonathan Eskew
@jeskew
@Zxurian How big is the file you’re uploading? >= 16MB?
Zxurian
@Zxurian
@jeskew some are, yes
in fact most of them
Jonathan Eskew
@jeskew
Those will be handled as multipart uploads
So it looks like all uploads will start
But multipart uploads may not finish before the end of a script
this is the test I was running:
$s3 = new \Aws\S3\S3Client([
    'region' => 'us-west-2',
    'version' => 'latest',
    'debug' => true,
]);

$uploads = [
    'small' => $s3->uploadAsync(
        'testing-testing-testing',
        'small-file-async-upload',
        fopen(__FILE__, 'r')
    ),
    'large' => $s3->uploadAsync(
        'testing-testing-testing',
        'large-file-async-upload',
        fopen(__DIR__ . '/phparchitect-2015-06.pdf' , 'r'),
        'private',
        [
            'before_upload' => function (\Aws\CommandInterface $command) {
                echo "uploading chunk {$command['PartNumber']}\n";
            }
        ]
    )
];

echo "uploading\n";
\GuzzleHttp\Promise\queue()->run();
The second upload is large enough to trigger a multipart upload
Which is handled by a Guzzle coroutine
And coroutines might not be complete before the call to \GuzzleHttp\Promise\queue()->run() finishes
Zxurian
@Zxurian
hm...
creating a test script now
Jonathan Eskew
@jeskew
does that sound like it’s the same error you’re seeing?
Zxurian
@Zxurian
well mine's not erroring out, it's just never completing
Jonathan Eskew
@jeskew
right
but does that fit the general characteristics of the issue you’re facing?
the script terminates and uploads still haven’t finished
Zxurian
@Zxurian
I'm getting the promise from the uploadAsync(), setting a then() on the promise, then looping waiting for a value to be set from the then(), so my script never terminates
it just keeps waiting for it but never completes, and chekcking separately on S3, nothing ever gets uploaded
Jonathan Eskew
@jeskew
Ok
Guzzle doesn’t have a built in event loop, so coroutines and functions registered with ->then need some prompting
This can either be by calling ->wait, which will unwind a full promise stack
Zxurian
@Zxurian
hm...so if I'm just issuing ->uploadSync() like so
Jonathan Eskew
@jeskew
or by manually ticking the loop
Zxurian
@Zxurian
$promise1 = $S3->uploadAsync('tantor-books', 'Test/test.xml', fopen('/mnt/olympus/Books/0087_CoalHuman/Download/0087_CoalHuman.xml', 'r'));
$promise1->then(function() use ($uploading) {
    $uploading --;
});
it won't automatically start uploading?
Jonathan Eskew
@jeskew
It will
but the upload will kick off additional promises
Zxurian
@Zxurian
that's what I thought. okay, let me run through some additional tests quick
Jonathan Eskew
@jeskew
one of the core developers of React wrote an integration
where the React loop will tick the Guzzle loop
and that can achieve something close to what you’re describing
Zxurian
@Zxurian
thanks, I'll take a look at it
so I'm working with this basic example
```php
<?php

$uploading = 0;

$AWS = new Aws\Sdk([
    'version'    => 'latest',
    'region'    => 'us-east-1',
    'credentials'    => [
        'key'        => '***',
        'secret'    => '***',
    ]
]);

$S3 = $AWS->createS3();

echo "beginning first promise\n";
$uploading ++;
$promise1 = $S3->uploadAsync('tantor-books', 'Test/test.xml', fopen('/mnt/olympus/Books/0087_CoalHuman/Download/0087_CoalHuman.xml', 'r'));
$promise1->then(function() use ($uploading) {
    $uploading --;
});

echo "beginning second promise\n";
$uploading ++;
$promise2 = $S3->uploadAsync('tantor-books', 'Test/test.zip', fopen('/mnt/olympus/Books/0087_CoalHuman/Download/0087_CoalHuman.zip', 'r'));
$promise2->then(function() use ($uploading) {
    $uploading --;
});

echo "beginning loop\n";
while ($uploading != 0) {
    echo 'uploading: '.$uploading."\n";
}
Jonathan Eskew
@jeskew
For that use case, I think it would be easier to collect promises and wait on an array of them
Zxurian
@Zxurian
and just running that, the script will never exit, as it will loop continuously on the last section, as the $uploading value never changes, because the then() functions aren't firing, because the files never upload. I've also let it run and checked externally to see if the files ever showed up on the S3 bucket, and they didn't, which leads me to believe that the uploadAsync() method isn't firing when called
Jonathan Eskew
@jeskew
That while loop won’t give anything a chance to upload
Zxurian
@Zxurian
but you said earlier that issuing an uploadAsync() starts immediately