scireum/OpenSource

General discussions and chit chat about OpenSource software made by scireum - http://www.scireum.de

draim

@draim

Sure, I can file an issue in GH. I'll simply sum up what I've got so far, more or less what is written here.

Thanks for the help anyway!

Andreas Haufler

@andyHa

no worries

Chad Birch

@chadbirch

I'm having some issues setting Content-Length in a HEAD response from S3Ninja. In the S3Controller, I add the following line to the response in S3Controller.java/GetObject:541:

response.addHeader(HttpHeaderNames.CONTENT_LENGTH, object.getFile().length());

I added some console logging for verification. When I rebuild and execute a HEAD request, the console reports the correct 'content-length', but my response header has 'content-length -> 0'. I'm using Postman. This is occurring before:

response.status(HttpResponseStatus.OK);

So it should be added to the response before it is complete. Any ideas what happening? Is it getting reset somewhere?

Zakaria Amine

@zak905

Hello everybody, I am having an issue with S3 Ninja. I extended the AWS SDK client, and tweaked so that it uploads to my S3 server

*S3 ninja

The problem is that I receive images in binary format, so they are not recognized by the computer as valid Jpeg

is there anyway to debug the requests received by the S3 ninja server

or maybe run the server in verbose mode, intead of the background

any help is much welcomed

Andreas Haufler

@andyHa

you can clone the github repo - if you didn’t already

fire up intellij and start s3ninja das normal application in the debugger

„IPL“ is the main class

then you can place breakpoints as you desire^^

Zakaria Amine

@zak905

@andyHa Thanks I will give it a try

Andreas Haufler

@andyHa

no worries, let un know if you need any further help

Zakaria Amine

@zak905

Here is the utility that extends AmazonS3Client that I created: https://gist.github.com/zak905/b00dcf897664f72c0bd7cb1248dabd1d

it can be used with S3Ninja and any other s3 mock

Hope this comes handy

for any people willing to mock s3 with AWS SDK

satyajitburman

@satyajitburman_twitter

Hi I am getting following error

docker_1 | Exception in thread "main" java.lang.reflect.InvocationTargetException
docker_1 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
docker_1 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
docker_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
docker_1 | at java.lang.reflect.Method.invoke(Method.java:498)
docker_1 | at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
docker_1 | at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
docker_1 | at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
docker_1 | at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
docker_1 | Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Not Found (Service: Amazon S3; Status Code: 404; Error Code: 404 Not Found; Request ID: null), S3 Extended Request ID: null
docker_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588)
docker_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258)
docker_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
docker_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742)
docker_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716)
docker_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
docker_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
docker_1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
docker_1 | at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
docker_1 | at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4187)
docker_1 | at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4134)
docker_1 | at com.amazonaws.services.s3.AmazonS3Client.createBucket(AmazonS3Client.java:1021)
docker_1 | at com.amazonaws.services.s3.AmazonS3Client.createBucket(AmazonS3Client.java:959)
docker_1 | at org.application_health.health.HealthApplication.main(HealthApplication.java:38)
docker_1 | ... 8 more

My class is below:

@SpringBootApplication
@Configuration
@EnableAutoConfiguration
@EnableAdminServer
public class HealthApplication {

public static void main(
        String[] args) {
    SpringApplication.run(HealthApplication.class, args);

    BasicAWSCredentials credentials =
            new BasicAWSCredentials(
                    "AKIAIOSFODNN7EXAMPLE",
                    "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY");
    EndpointConfiguration endpointConfiguration =
            new EndpointConfiguration(
                    "http://docker-s3ninja:9444",
                    "us-east-1");
    AmazonS3ClientBuilder amazonS3ClientBuilder =
            AmazonS3ClientBuilder.standard();
    amazonS3ClientBuilder.withEndpointConfiguration(endpointConfiguration);
    amazonS3ClientBuilder.enablePathStyleAccess();
    AmazonS3 amazonS3Client =
            amazonS3ClientBuilder
                    .withCredentials(
                            new AWSStaticCredentialsProvider(credentials))
                    .build();

    amazonS3Client.createBucket("mybucket");

}

}

Why I am getting it, what is the issue.

Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Not Found (Service: Amazon S3; Status Code: 404; Error Code: 404 Not Found; Request ID: null), S3 Extended Request ID: null

satyajitburman

@satyajitburman_twitter

Any one facing this issue?

satyajitburman

@satyajitburman_twitter

Hi, I've finally returned to experimenting with S3 ninja. Is there a way to make it get the bucket name from the host name instead of from the path? We're (sadly) using s3cmd and it does not support "path style access" as has been suggedted.

how can I use sub-domain style access. like mybucket.localhost:9444/s3 or mybucket.s3.localhost:9444

Andreas Haufler

@andyHa

currently only path style access is supported

AWSCredentials credentials = new BasicAWSCredentials(s3AccessKey, s3SecretKey);
ClientConfiguration config = new ClientConfiguration().withSocketTimeout(LONG_SOCKET_TIMEOUT);
if (useLegacySigner) {
config = config.withSignerOverride("S3SignerType");
}
AmazonS3Client newClient = new AmazonS3Client(credentials, config);
newClient.setS3ClientOptions(new S3ClientOptions().withPathStyleAccess(true));
newClient.setEndpoint(s3EndPoint);

http://localhost:9444/s3 would be the endpoint

    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk-s3</artifactId>
        <version>1.10.12</version>
    </dependency>

we use that version (aws is quite picky)

Gustavo Ponce

@gustavoponce7

HI All, Have you faced the next exception? Caused by: java.security.NoSuchAlgorithmException: Algorithm HmacSHA1 not available

Gustavo Ponce

@gustavoponce7

I have resolved this exception updating my jdk from "1.8.0_151" to "1.8.0_162"

charlie4zhang

@charlie4zhang

Hi, I have seen this issue, could someone give me a hand?

I have tried to clone the Sirius-kernel source code and do "mvn clean install", and see unit test failure shown:

BOMReaderTest.readArray10BOM:58 expected:<5> but was:<8>
BOMReaderTest.readArray1BOM:43 expected:<72> but was:<239>
BOMReaderTest.readArray2BOM:51 expected:<72> but was:<239>

My system set up is:

mvn -version
Apache Maven 3.5.3 (3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T12:49:05-07:00)
Maven home: C:\java\mvn\apache-maven-3.5.3
Java version: 1.8.0_151, vendor: Oracle Corporation
Java home: C:\java\jdk1.8.0_151\jre
Any idea what I am missing ?

I can change the test expected value to get the test pass but my question is are these reflect some real problem in the java class code?

Saurabh Valsangkar

@saurabhvalsangkar

hi - This is a great project and I am trying to use it for testing in development environment. When I configured it and tried to add a bucket, I am getting below error. Am I missing something?

sirius.kernel.health.HandledException: An unexpected exception occurred: POST (java.lang.IllegalArgumentException)
at sirius.kernel.health.Exceptions$ErrorHandler.handle(Exceptions.java:193)
at sirius.kernel.health.Exceptions.handle(Exceptions.java:350)
at sirius.web.http.DispatcherPipeline.handleInternalServerError(DispatcherPipeline.java:100)
at sirius.web.http.DispatcherPipeline.dispatch(DispatcherPipeline.java:91)
at sirius.web.http.DispatcherPipeline.lambda$dispatch$2(DispatcherPipeline.java:87)
at sirius.web.http.WebDispatcher.dispatch(WebDispatcher.java:102)
at sirius.web.http.DispatcherPipeline.dispatch(DispatcherPipeline.java:83)
at sirius.web.http.DispatcherPipeline.lambda$dispatch$2(DispatcherPipeline.java:87)
at sirius.web.http.WebDispatcher.dispatch(WebDispatcher.java:102)
at sirius.web.http.DispatcherPipeline.dispatch(DispatcherPipeline.java:83)
at sirius.web.http.DispatcherPipeline.lambda$dispatch$2(DispatcherPipeline.java:87)
at sirius.web.http.WebDispatcher.dispatch(WebDispatcher.java:102)
at sirius.web.http.DispatcherPipeline.dispatch(DispatcherPipeline.java:83)
at sirius.web.http.DispatcherPipeline.lambda$dispatch$2(DispatcherPipeline.java:87)
at sirius.web.http.WebDispatcher.dispatch(WebDispatcher.java:102)
at sirius.web.http.DispatcherPipeline.dispatch(DispatcherPipeline.java:83)
at sirius.web.http.DispatcherPipeline.lambda$dispatch$1(DispatcherPipeline.java:73)
at sirius.kernel.async.ExecutionBuilder$TaskWrapper.run(ExecutionBuilder.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: POST
at ninja.S3Dispatcher.listBuckets(S3Dispatcher.java:264)
at ninja.S3Dispatcher.dispatch(S3Dispatcher.java:166)
at sirius.web.http.WebDispatcher.dispatch(WebDispatcher.java:101)
at sirius.web.http.DispatcherPipeline.dispatch(DispatcherPipeline.java:83)
... 17 more

Jin Bal

@jinbal

Hi
I see that from a commit on July 20th:
scireum/s3ninja@663da1b
That you've moved to using RFC date formatting in xml responses yet i can see in the aws-sdk-java (1.11.382) that it is still using ISO dates to parse responses :

        else if (in("ListBucketResult", "Contents")) {
                if (name.equals("Key")) {
                    lastKey = getText();
                    currentObject.setKey(decodeIfSpecified
                                    (lastKey, shouldSDKDecodeResponse));
                } else if (name.equals("LastModified")) {
                    currentObject.setLastModified(
                            ServiceUtils.parseIso8601Date(getText()));
                } else if (name.equals("ETag")) {

this is causing the client (1.11.382) calls to s3Client.listObjects(bucket) to fail with parsing errors

Andreas Haufler

@andyHa

Jason

@nycjay

Hello

Got a question about s3ninja

neaos

@neaos

hello, anybody is online？

I wanna ask a easy question：what is the parsii and how to use it to Calculation Mathematical Expressions，for example ,2-4*5

neaos

@neaos

why do we need to evaluator for mathematical expressions

I was trying to integrate mathics, but I found parsii by mistake.

neaos

@neaos

@andyHa

<dependency>
<groupId>com.scireum</groupId>
<artifactId>parsii</artifactId>
<version>1.5</version>
</dependency>

The newest jar is 1.5？

Where communities thrive

People

Repo info

Activity