discussion forum for www.deeplabcut.org; also the forum for posting questions: https://forum.image.sc/tags/deeplabcut
and the training results is :
Results for 20000 training iterations: 95 1 train error: 1.57 pixels. Test error: 7.43 pixels.
With pcutoff of 0.1 train error: 1.57 pixels. Test error: 7.43 pixels
Next i continue the following steps, and finish all the procedures.
I tried to open the filtered video produced by the program and i found there are several problems:
one is the label animal is not the one i labeled for training before(i have many animals within one video and most of animals are marked with number tag) and the label objects jump to another animals with another number tag or even jump to animals without number tag, and sometimes even jump to the wall of the space where the animals live.
the second is the results of the plot_poses, they alĺ look ugly:
whats the best way to deal with nvidia-docker not being available on ubuntu 18.04? (only nvidia-docker2)
tried loading from a repo with the older package but still getting
The following packages have unmet dependencies:
nvidia-docker : Depends: sysv-rc (>= 2.88dsf-24) but it is not installable or
file-rc (>= 0.8.16) but it is not installable
E: Unable to correct problems, you have held broken packages.
and can't figure out how I got around this in the past
@tfrostig
Under: "Create a New Project" heading here:
https://github.com/AlexEMG/DeepLabCut/blob/master/docs/UseOverviewGuide.md
The function to invoke multiple videos is shown to be:deeplabcut.create_new_project(`Name of the project',`Name of the experimenter', [`Full path of video 1',`Full path of video2',`Full path of video3'], working_directory=`Full path of the working directory',copy_videos=True/False)
So lets say you already have ProjectA, ProjectB, and Project C -- but you want to merge them into Project1. If you already have some of the frames of other videos labeled, they will be in ./ProjectA/labeled-data/<Video1> and ./ProjectA/labeled-data/<Video1>_labeled. So you can copy those two directories (the ones inside./ProjectA/labeled-data/ ) to ./Project1/labeled-data/. Then do the same for ./ProjectB/labeled-data/ and ./ProjectC/labeled-data/.
Assuming you have enough labeled frames covering most of the unique cases in your footage, you can skip deeplabcut.extract_frames(), deeplabcut.check_labels(), and deeplabcut.label_frames() to go straight to deeplabcut.create_training_dataset().
P.S., I don't think ./ProjectA/labeled-data/<Video1>_labeled is actually used by the software - i think it's generated for human convenience only when deeplabcut.check_labels() is ran. The actual labels are stored in ./ProjectA/labeled-data/<Video1>/CollectedData_<User>.csv and/or ./ProjectA/labeled-data/<Video1>/CollectedData_<User>.h5
whats the best way to test that my container has the ability to use GPU for training?
Inside docker, nvcc / nvisida-smi seems to be working fine, e.g.
ubuntu@4d3df5a62144:/work$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Sep__1_21:08:03_CDT_2017
Cuda compilation tools, release 9.0, V9.0.176
However, deeplabcut.train_network(config_path,shuffle=1,trainingsetindex=0,gputouse=0,max_snapshots_to_keep=1,autotune=False,displayiters=None,saveiters= None, maxiters=None)
results in CPU max loading, and no raise in GPU memory utilization (sticks at 0.4GB / 1GB regardless if training or not). Listing processes in nvidia-smi is not supported by my graphics card, however my GPU does seem to support CUDA 9 with a compute capability of 3.0
CUDA V9.0.176
Nvidia-driver-390.129
@tvajtay
Doesn't look like I've got TF running on GPU. Inside the container, running the following in ipython
import tensorflow as tf
...: with tf.device('/gpu:0'):
...: a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='
...: a')
...: b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='
...: b')
...: c = tf.matmul(a, b)
...:
...: with tf.Session() as sess:
...: print (sess.run(c))results inInvalidArgumentError (see above for traceback): Cannot assign a device for operation 'MatMul_1': Operation was explicitly assigned to /device:GPU:0 but available devices are [ /job:localhost/replica:0/task:0/device:CPU:0 ]. Make sure the device specification refers to a valid device.
@tvajtay thanks for teaching me a new command with conda list.
between then and now I ran conda install tensorflow-gpu=1.8.0 so it definitely shows up at the moment...
I'm not sure if it was there earlier; I was using the https://github.com/eqs/DeepLabCut-Docker/blob/master/docker/Dockerfile (eqs) docker image. If you're familiar with it, you may know whether it had tf-gpu installed.
conda install tensorflow-gpu=1.8.0 inside the container I still have the same issue when I repeat the test above (print (sess.run(c)))