by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • May 29 06:21
    aliutkus synchronize #40
  • May 29 06:21

    aliutkus on torchfilters

    docs (compare)

  • May 29 05:48
    aliutkus edited #40
  • May 29 05:48
    aliutkus edited #40
  • May 29 05:35
    aliutkus synchronize #40
  • May 29 05:35

    aliutkus on torchfilters

    fixed bug in filtering and the … fixed tests (compare)

  • May 29 04:36
    aliutkus synchronize #40
  • May 29 04:36

    aliutkus on torchfilters

    removed the itertools.product (compare)

  • May 29 04:35
    aliutkus synchronize #40
  • May 29 04:35

    aliutkus on torchfilters

    changed the env name for travis (compare)

  • May 28 13:57
    aliutkus synchronize #40
  • May 28 13:57

    aliutkus on torchfilters

    removed the dataloaders for bat… added resampling in utils, and … modified eval and test for the … (compare)

  • May 26 07:52
    aliutkus synchronize #40
  • May 26 07:52

    aliutkus on torchfilters

    added the and switches (compare)

  • May 22 07:49

    faroit on dev

    fix audio loading for soundfile… (compare)

  • May 20 22:08
    aliutkus synchronize #40
  • May 20 22:08

    aliutkus on torchfilters

    repaired eval for latest change… bug in preload=False and build_… (compare)

  • May 20 21:36
    aliutkus synchronize #40
  • May 20 21:36

    aliutkus on torchfilters

    work in progress on filtering first working version with back… (compare)

  • May 19 18:22
    faroit closed #42
jhm0799
@jhm0799
I'll try it without that one
Nope
Removing those 2 parameters still gives an error about CUDA
RuntimeError: CUDA out of memory. Tried to allocate 158.00 MiB (GPU 0; 2.00 GiB total capacity; 1.27 GiB already allocated; 145.36 MiB free; 21.81 MiB cached)
jhm0799
@jhm0799
Tried this:
python -m demucs.separate -d cpu -n demucs c:\walk.wav -o c:\output
"TypeError: argument of type 'WindowsPath' is not iterable
jhm0799
@jhm0799
I give up. Can't figure this out

I tried this too:

python -m demucs.separate -n demucs -o output --models models -d cpu tracks walk.wav

Maybe I'm not doing it right

jhm0799
@jhm0799
I changed python3 to python since it says python3 isn't a valid command
Fabian-Robert Stöter
@faroit
please report this to the demucs repo. I think they would love some feedback
I was able to run the cpu version without any problems
jhm0799
@jhm0799
Will do. Was it on Windows?
Fabian-Robert Stöter
@faroit
OS X
jhm0799
@jhm0799

This error appears just before the WindowsPath one:

needquote = (" " in arg) or ("\t" in arg) or not arg

I think I'm mis-typing the command or something

Ah alright

Fabian-Robert Stöter
@faroit

Hi open-unmix users. Just a quick note here:

Since the release of open-unmix in September 2019, the source separation landscape has evolved regarding pre-trained models.

Open-unmix was designed as a baseline to make it easier to do source separation research. Now, spleeter has been made available and got much popularity since its release. We believe it is a good implementation for users to apply source separation in a signal processing pipeline.

For this reason, we decided to not release the tensorflow implementation of open-unmix publicly, but rather to make it available for commercial applications.

We continue to improve and update the PyTorch release and we believe that in terms of academic research, this follows the current trend in the community to vastly favor this framework.

Russell Izadi
@russellizadi_twitter
Hi, I just started exploring the unmix code and I have a question:
Since a model is needed for each target, how do you manage to deal with a relatively large number of sources? For example, consider the separation of a mixture of two mnist images. Do you estimate 10 separate models?!
I think open-unmix is mainly developed for music separation but it could be perfect to add more features so more separation tasks could be performed using this clean and organized repo.
Fabian-Robert Stöter
@faroit
@russellizadi_twitter yes, we believe that training separate models is a good tradeoff between efficiency and performance. Joint models may be a bit better in performance but many data sets are unbalanced when it comes to the number of sources. Therefore it makes sense to train separate models.
Aadi Bajpai
@aadibajpai
In a signal processing pipeline, would you recommend demucs or spleeter if I only really need to work with vocals?
Currently using umx, of course.
Fabian-Robert Stöter
@faroit
I depends on what you need
Well, they all have their pros. We designed open unmix to serve as a baseline for researchers not to beat SOTA. Spleeter serves end users that want to use separation in their pipeline. Demucs
represents a new line of research for end-to-end source separation.
so if you don't have your own training data set and you are okay with the quality of <11 kHz, you go with spleeter
Aadi Bajpai
@aadibajpai
We're basically trying to play with lyrics-to-audio alignment and vocal isolation is the first step there so any errors just propagate further. UMX has been better than whatever we had tried before but I feel we should try spleeter once in the pipeline. I was a bit confused about that quality loss thing initially.
Russell Izadi
@russellizadi_twitter

@faroit I don't understand why joint models may have better performance! Only one model with 1/4 of parameters of 4 models in total, should not perform better. My understanding is the trade-off is other way around. I mean a single model is more efficient (smaller in size) but has worse performance.

For example, you need 10 models for separating mixtures of 10 sources, which is not efficient, the overall performance is better than a single model (with the same size as one of those 10 models).

And a question, what are the best practices for defining/adding new models in a project like this? Do you just change the OpenUnmix model or define a new model and change the train/test/... files accordingly?
Fabian-Robert Stöter
@faroit
@russellizadi_twitter by single model I mean a joint model that has e.g. 4 times the number of parameters and the 4 outputs (heads) are joint at the end to be able to train with multiple sources jointly
@russellizadi_twitter concerning improving the performance, I would suggest to start on the model side. The training code is very simple by design and can easily be adopted. I would suggest to stick with out data loaders since we put a lot of work into these to make them flexible and efficient. Also see our documentation here: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/extensions.md
PeturBryde
@PeturBryde
Hello! I'm wondering whether the umx and umxhq models are both trained with the default parameters of train.py, and in particular whether they both use --bandwidth 16000? What would be the benefits of training with a higher bandwidth setting?
Fabian-Robert Stöter
@faroit
@PeturBryde there a some differences to the defaults. You'll find the exact training configuration here: https://zenodo.org/record/3370486 and https://zenodo.org/record/3370489 in the training-json-logs.zip file
@PeturBryde concerning the bandwidth argument: this does only affect the internal parameters of the model. UMXHQ can predict spectra with >16 kHz even though most layers only use up to 16khz as the model does bandwidth extension internally. We didn't observe better quality when increasing --bandwidth to something like 22050. Hope that helps
PeturBryde
@PeturBryde
@faroit Thank you, that's very helpful!
jhm0799
@jhm0799
I actually used denucs for something lol
Nothing commercial of course, but still something nonetheless
Spleeter failed to give good results so i tried demucs. It worked a lot better
jhm0799
@jhm0799
A song is out there which has no instrumental and I wanted one. None of the tools I've used have been able to remove the vocals as good as I wanted it, so i decided to resequence the song (or attempt to). This failed as its a very complex song, so I only re-sequenced some of it (bass, a few synths) and I layered the drums and "other" stem (extracted with denucs). That makes it sound great
Denucs fails to grab reverb it seems
jhm0799
@jhm0799
The bass synth really throws off the drum extractor on Spleeter lol
jhm0799
@jhm0799
Anyways, I was in for a surprise when I ran Demucs on a track which a friend of mine made. It extracted a bass part which didn't exist in the original track. Meaning: It extracted a different sequence of notes... It must've guessed a new melody or something. I asked my friend about it and he said "It most definitely messed up". haha
It sounds like a proper bassline though
methevoz
@methevoz
Hi, I'm doing a Master thesis on musical source separation and have been using UMX a lot in my research.
First of all, thanks a lot for the amazing work you did and for providing an open-source reference implementation of such quality to the source separation community.
Recently I've been trying to measure the impact of the amount of training data by using bigger private datasets (good quality, ~10 times bigger than MUSDB18). I trained without messing too much with the default parameters of UMXHQ.
Surprisingly it looks like I get a drop in performance because of a significant overfit on the training data. I've thought about what could be the cause and considered tweeking the weight/learning rate decays, or the dropout rate.
I don't have a lot of leeway for playing with parameters because of the time it takes to train a model on datasets of this scale, so I was wondering if you could provide some insight and advice for dealing with overfit since you have experience with that issue, and with training on bigger datasets (with UMXPRO)?
Fabian-Robert Stöter
@faroit
@methevoz we are currently preparing a similar work where we evaluate the scale of open-unmix. I have done that with our big internal dataset of 500h of clean separated stems and we get significantly better results. It does certainly make sense to modify the learning rates scheduling but even more important is so increase the number of parameters for the model.
by the way, can you reveal any details about your dataset? is it commercial pop music? Because we observe a big bias on the kind of musical tracks. They would have the be of similar genres as MUSDB18 (rock+pop) to improve the scores
methevoz
@methevoz
@faroit Thanks for the quick answer! I see, do you know what would be a good strategy for exploring these parameter tweaking without spending weeks on grid search & cross validation? Like an intuitive scale on how much to increase/decrease certain parameters based on the size of the dataset only.
As for the dataset itself, it's a mix of a few popular songs from the last 50 years so there are multiple genres in there (rock, pop and others) but yes there is a lot of commercial pop music indeed.
Note though that the overfit I observe is with the validation set of my private dataset, I haven't tested the models on MUSDB18 yet.
jhm0799
@jhm0799
I've really gotten to like Demucs. I'd be using UMX for my stuff but I can't set it up on Windows it seems... Anyways, I've made at least a few things using Demucs. Quite revolutionary :)
methevoz
@methevoz
@faroit Hello, sorry if it is a silly question but do you have a piece of code for displaying the mag spectrograms input to / output by Open-Unmix pytorch, i.e., the torch tensors of shape (nb_frames, nb_samples, nb_channels, nb_bins)? I would love to be able to visualize those and compare them with the specs of my own network.
Fabian-Robert Stöter
@faroit
I suggest you print just the one channel and just on sample in a batch to make things easier
Have a look at our tutorial notebooks https://sigsep.github.io/tutorials/ for code examples
vijai9111995
@vijai9111995
I want to do spectral unmixing using hyperspectral images in python. Like what to extract the spectral reflectance of each feature for ex. land, water body etc... which is present in the image. how to do unmixing?? what command should i have to use it?
Fabian-Robert Stöter
@faroit
I am not really familiar with spectral images. This package deals with audio unmixing, I'm not sure if its helpful for you