by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Hyungui Lim
    @hyungui
    Thanks!
    SP Mohanty
    @spMohanty

    The deadline for submission of your models packaged as docker containers for round-2 has been extended to April 8th, 2018.

    The grading server will expect the code to be a Binder compatible repository.

    Predictions will be made on an arbitrary number of mp3 files of at most 30seconds each.
    During the execution of the container, All the mp3 files will be mounted at the location : /crowdai-payload.
    Execution of your container will be initiated by executing /home/run.sh /crowdai-payload.

    More details of how to package your code as Binder compatible repositories, please read the documentation :

    Round2_packaging_guidelines.md.
    Round2_submission_guidelines.md.

    During the runtime, the container will not have access to the external internet, and will have access to :

    • 1 Nvidia GTX GeForce 1080 Ti (11 GB GDDR5X)
    • 5 cores of an Intel Xeon E5-2650 v4 (2.20-2.90 GHz)
    • 60 GB of RAM
    • 100 GB of disk

    and a timeout of 10 hours.

    Note that, the results from your submitted containers will be announced at the end of Round-2.

    cc. @mdeff

    AlgoHunt
    @AlgoHunt
    Is it necessary to announcen a team, as we dont see any team up buttom or something like it
    SP Mohanty
    @spMohanty
    In case of teaming up, please just send us an email and letting us know
    Apart from that, have a single repository for your submission(s), and remember to add the usernames of all your team mates in the authors params of crowdai.json (https://github.com/crowdAI/crowdai-musical-genre-recognition-starter-kit/blob/master/Round2_submission_guidelines.md#meta-data)
    Minz Won
    @minzwon
    I have questions regarding the GPU usage. Did you check if the docker image built with the provided instruction works well with a GPU? I'm not sure if repo2docker sets up the cudnn together. If you checked its working, is it compatible for the Theano backend as well? Also, for the GPU usage, I think we should run nvidia-docker instead of docker.
    SP Mohanty
    @spMohanty
    @minzwon : Good point, and yes, you will get access to 1 GPU, and during orchestration we will be using nvidia-docker during the execution of the container. I forgot to add this to the submission+packaging guidelines. Do you want to add a section about that ? Happy to merge in a pull request.
    Else, I will try to add a section on it later today, or early tomorrow
    SP Mohanty
    @spMohanty
    @minzwon : I have added a small section on nvidia-docker to the guidelines. Thanks for the reminder
    Minz Won
    @minzwon
    @spMohanty Thank you! I have one more question. Is it okay to build a docker image without using repo2docker? Since I have a pre-built docker image that I am using, and it's on the docker hub, it would be more convenient to submit using that image.
    SP Mohanty
    @spMohanty
    @minzwon : Yes, you can also include a Dockerfile instead. but we will still be using repo2docker to build to for consistency.
    repo2docker will prioritise a Dockerfile if it sees one
    Hyungui Lim
    @hyungui
    Hi, how can I check my packaging and submission are OK to be evaluated? I think I followed all the guides, but I'm not sure it's gonna work fine..
    Benjamin Murauer
    @bmurauer

    Hi, thanks for the detailed instruction page! I was able to produce a binder-compatible repository, I think ;)
    I still have 2 small questions, mainly because I don't have much experience with docker:

    Thank you for your efforts! :)

    SP Mohanty
    @spMohanty
    @hyungui @bmurauer : Hi guys, unfortunately the automate pipeline for building is coughing a little bit. But I will manually test your submissions and get back to you.
    Also regarding the -v flag, dont worry about it. We will run it at our end.
    and yeah, the output is stored in the container, and we extract it, and validate it. then the container is destroyed (and hence the output)
    chumbalov
    @chumbalov
    @spMohanty Hello, we have problems with building our repository. Could you check it manually?
    pushnyakov
    @pushnyakov
    We finally managed to build docker image locally, but please let us know if you spot any problems while running it!
    pushnyakov
    @pushnyakov
    @spMohanty
    SP Mohanty
    @spMohanty
    @pushnyakov @chumbalov : Please do create your repositories on gitlab.crowdai.org even if its not ready, and provide some documentation for running it. We promise to spend a reasonable amount of time to get it to run. If it has complicated dependencies or throws errors we do not understand, we will not be able to get into a debug cycle for the same.
    Please do this by the 10th April (Tuesday) 23:59 UTC. + 2
    pushnyakov
    @pushnyakov
    @spMohanty We have already created the repo: https://gitlab.crowdai.org/gg12/WWWMusicalGenreRecognitionChallenge
    And documentation is inside README.md
    Michaël Defferrard
    @mdeff
    @mimbres We have been trying to run your code (and to reach you) without success. Please take a look at the issues here:
    https://gitlab.crowdai.org/mimbres/WWWMusicalGenreRecognitionChallenge/issues
    We are presenting the results at the WebConference this Friday, it would be a pity if we could not include yours, as they seem very impressive on round1. Thanks a lot!
    Sungkyun Chang
    @mimbres
    @mdeff
    File "util/audio_processing_test_np_sep_spec_MlutiThread.py", line 74, in preprocess_func
    test_ids = org_test_ids[t[0]:t[-1]]
    IndexError: index 0 is out of bounds for axis 0 with size 0
    @mdeff
    @mimbres I could reproduce same error if the test files were not .mp3. Please modify Line 47 of Util/audio_processing_tesp_np_sep_spec_MultiThread.py :
    Sungkyun Chang
    @mimbres
    @mdeff org_test_ids = sorted(glob.glob(fma_testset_filedir + '*.mp3'))
    @mdeff If your files were not mp3 format, org_test_ids will be empty. Then we will get indexError. Please modify Line47 '.mp3' to '.<YOUR_AUDIO_FORMAT>' or just '.*'.
    Sungkyun Chang
    @mimbres
    Also, please make sure that you have more than 50 test files in the directory. Because in our code, each single thread requires at least 50 files to pre-process.
    Currently, I updated the line 47 with replacing '.mp3' to '.*'. Please make sure that your contains only audio files, and retry.
    Michaël Defferrard
    @mdeff
    Thanks @mimbres for your answers. The files are all mp3s, and it's 3,000 files. So it should not be those issues.
    Moreover, my colleague @spMohanty ran your code again and he's still getting the same error: https://gitlab.crowdai.org/mimbres/WWWMusicalGenreRecognitionChallenge/issues/2#note_44
    SP Mohanty
    @spMohanty
    I also suspected it could be because of this issue, but that also doesnt solve the problem.
    @mimbres: Can you confirm if it runs perfectly at your end ? I would be surprised if the same error doesnt happen at your end
    assuming that the test data inside data/round_1/ folder, this is what I do :
    export IMAGE_NAME="submission_image_mimbres"
    export CONTAINER_NAME="container_name_mimbres"
    
    repo2docker --no-run \
      --user-id 1001 \
      --user-name crowdai \
      --image-name $IMAGE_NAME \
      --debug .
    
    CUDA_VISIBLE_DEVICES=1 docker run \
      -v `pwd`/data/round_1/:/crowdai-payload \
      -e TEST_DIRECTORY='/crowdai-payload/' \
      -e OUTPUT_PATH='/tmp/output.csv' \
      --name $CONTAINER_NAME \
      -it $IMAGE_NAME \
      /home/crowdai/run.sh
    
    docker cp $CONTAINER_NAME:/tmp/output.csv output_round_1.csv
    SP Mohanty
    @spMohanty
    Okay, maybe I found the issue. It was probably this one : https://gitlab.crowdai.org/mimbres/WWWMusicalGenreRecognitionChallenge/commit/65325d9143c52fc1e31e21ae0ea1fc6515b61aa3
    The ffmpeg dependency was missing
    so overall it was a combination of the bug in glob.glob not using os.path.join, and this lack of ffmpeg dependency in the conda environment.
    SP Mohanty
    @spMohanty
    Screen Shot 2018-04-27 at 02.10.11.png
    SP Mohanty
    @spMohanty
    @mimbres : How long would you expect it to run for 3000 files on 48 cores ?
    Sungkyun Chang
    @mimbres
    @spMohanty My bad! Sorry for your inconvenience. I also created a docker by following the your guide, and it was successful in my system. Anyway you seem to solve the issue now, right? FYI, the preprocessing of 12 cores-AMD 1700 with 35,000 songs took 3-4 hours.
    Sungkyun Chang
    @mimbres
    @spMohanty After preprocessing done, genre prediction of 35,000 songs (35,000 X 15 segments) took 25-30 min with a single 1080ti GPU and NVMe SSD. So I expect 3,000 songs in your test would finish at least within 1 hour. Finally, if you still can't solve the issue, please check email box(2018. 04. 09). I sent you a link of pre-built Docker image. You can directly download it from the Docker Cloud. You may need to replace run.sh with updated one(from gitlab) and run.
    pushnyakov
    @pushnyakov
    @spMohanty @mdeff Dear organizers, could you please tell us where will the final results be available?
    pushnyakov
    @pushnyakov
    Guys, here is our solution which was on the 4th place on public LB https://github.com/pushnyakov/WWWMusicalGenreRecognitionChallenge
    Michaël Defferrard
    @mdeff
    Dear @pushnyakov, we'll let you know here and on the forum where to find them. We are still missing one number...
    BTW we'll also make all your repositories public at the same time, so that you'll be able to find out how each team solved the challenge. :)
    Michaël Defferrard
    @mdeff

    Dear all,

    it's our great pleasure to announce that the winners of the second round is the team formed by Jaehun Kim (@jaehun) and Minz Won (@minzwon)! As promised, you are invited to present your results at the Applied Machine Learning Days at EPFL in January 2019. Congrats! :)

    You'll find all the scores on the leaderboard of the second round. The 6 systems that have been submitted are now open-source, so you can inspect how others did it! You'll find links to their repositories on our starter-kit repository, along with a summary table of the results. Finally, you can find more results and a discussion on the slides used to announce them.

    We apologize for the delay. The main reason was that some of the submitted systems were not able to run properly, and we had to debug them with the authors. We didn't want any of you who made the effort to submit a self-contained system to be left out.

    Finally, thanks again to all the participants! It was a pleasure to organize this challenge for you, and we hope you had fun and learned while participating.

    Michaël, on behalf of the organizers, Mohanty, Sean, and Marcel.

    Minz Won
    @minzwon
    Thank you for the good news and your effort for the challenge :)
    Benjamin Murauer
    @bmurauer
    Congratulations! Were you able to get home well after the strikes?
    Minz Won
    @minzwon
    @bmurauer I just had to stay one more night in there but I enjoyed the city