These are chat archives for dgraph-io/dgraph

14th
Apr 2016
Pawan Rawal
@pawanrawal
Apr 14 2016 01:34
users ?
Manish R Jain
@manishrjain
Apr 14 2016 01:52
Category in discuss.
Pawan Rawal
@pawanrawal
Apr 14 2016 01:54
Ok sure
Pawan Rawal
@pawanrawal
Apr 14 2016 02:06
also if my loader process has crashed once and I restart it again , I get this
ERRO[0296] While converting to edge error=xid: m.032zq6 doesn't have any uid assigned. nq={Subject:m.032zq6 Predicate:type.object.name.pl ObjectId: ObjectValue:Między piekłem a niebem Label:} package=loader. I think uidassigner worked just fine. Any idea why this might be happening ?
Do I need to clear out something before starting the loader again.
Manish R Jain
@manishrjain
Apr 14 2016 02:06
Did you merge the uids?
Pawan Rawal
@pawanrawal
Apr 14 2016 02:07
Just loaded with numInstance =1
Manish R Jain
@manishrjain
Apr 14 2016 02:07
hmm... then this shouldn't happen.
You sure the uid assigner finished successfully? I'd suggest running it again
If you run it over the same uid directory, it would be faster.
Pawan Rawal
@pawanrawal
Apr 14 2016 02:08
Ok, how does program know where to pick uids from, because when they are merged, they go to final.uids whereas otherwise they are in uids/u0
Manish R Jain
@manishrjain
Apr 14 2016 02:09
you need to pass the uids directory.
Can you paste the loader command, you executed?
Pawan Rawal
@pawanrawal
Apr 14 2016 02:09
go build . && ./loader --numInstances 3 --instanceIdx 0 --rdfgzips $BENCHMARK_REPO/data/names.gz,$BENCHMARK_REPO/data/rdf-films.gz --uids ~/dgraph/uasync.final --postings ~/dgraph/p0 --numCpu 1 --stw_ram_mb 2500
Manish R Jain
@manishrjain
Apr 14 2016 02:10
Did you generate the uasync.final, or did you copy it over?
Pawan Rawal
@pawanrawal
Apr 14 2016 02:10
got it from benchmarks repo
Manish R Jain
@manishrjain
Apr 14 2016 02:10
Won't work anymore. Ashwin changed the way data is stored.
Pawan Rawal
@pawanrawal
Apr 14 2016 02:11
oh
Manish R Jain
@manishrjain
Apr 14 2016 02:11
You need to generate the uids.
I thought you were doing that, because you said above that your uidassigner worked fine...
Pawan Rawal
@pawanrawal
Apr 14 2016 02:11
Ok, which go in uids/u0 , so I need to pass this directory to the loader
if my uidassiger command was this
go build . && ./uidassigner --numInstances 1 --instanceIdx 0 --rdfgzips $BENCHMARK_REPO/data/rdf-films.gz,$BENCHMARK_REPO/data/names.gz --uids ~/dgraph/uids/u0 --stw_ram_mb 1500 --numCpu 1
Manish R Jain
@manishrjain
Apr 14 2016 02:13
Yes. Think about what you're doing here. So, you generated the uids for all the entities in the data, with the output going to ~/dgraph/uids/u0
Now, you need to pass these uids to the loader, so it can then load the data.
Pawan Rawal
@pawanrawal
Apr 14 2016 02:16
Ok, but in the documentation the destination is go build . && ./merge --stores ~/dgraph/uids --dest ~/dgraph/final.uids and the loader has uids directory as --uids ~/dgraph/uasync.final
that was confusing.
Manish R Jain
@manishrjain
Apr 14 2016 02:17
That's because it's merging the multiple uids instances into 1.
Pawan Rawal
@pawanrawal
Apr 14 2016 02:17
yup so shouldn't it load from the destination that is ~/dgraph/final.uids ?
Manish R Jain
@manishrjain
Apr 14 2016 02:17
You could run the merge step as well -- it wouldn't be so useful because you only have 1 instance.
oh.. I see what you mean. Looks like a documentation error
Fixing it
Pawan Rawal
@pawanrawal
Apr 14 2016 02:19
yup, kind of got me confused. Couldn't see the link between where uids where stored and where they were loaded from :worried:
Thanks :smile:
Manish R Jain
@manishrjain
Apr 14 2016 02:19
Yeah, sorry about that.
Pawan Rawal
@pawanrawal
Apr 14 2016 02:20
No problem :smile:
Manish R Jain
@manishrjain
Apr 14 2016 02:23
Fixed
Pawan Rawal
@pawanrawal
Apr 14 2016 02:25
great
Manish R Jain
@manishrjain
Apr 14 2016 02:36
btw, did you send out a change y'day?
@pawanrawal
I can't see it on reviewable
Pawan Rawal
@pawanrawal
Apr 14 2016 02:41
no, was stuck in testing it out. Thats why all the data loading and stuff.
Will send it out today hopefully.
Manish R Jain
@manishrjain
Apr 14 2016 02:43
Sure.
Manish R Jain
@manishrjain
Apr 14 2016 06:39
Hey @ashwin95r Do you need me to review your change?
Ashwin Ramesh
@ashwin95r
Apr 14 2016 07:44
Ah not now, need to do the changes Xiang mentioned
Ashwin Ramesh
@ashwin95r
Apr 14 2016 10:00
Hey @manishrjain , can you give me permission to push to benchmarks?
Manish R Jain
@manishrjain
Apr 14 2016 14:07
@ashwin95r done
Ashwin Ramesh
@ashwin95r
Apr 14 2016 14:35
Thanks!
Ashwin Ramesh
@ashwin95r
Apr 14 2016 16:00
Do we have a meeting tomorrow?
Pawan Rawal
@pawanrawal
Apr 14 2016 16:05
Yes, I think we have a weekly review meeting at 8:30.
That's what my calendar shows atleast :smile:
Prashanth Koppula
@koppula
Apr 14 2016 16:06
That’s on Monday
Team weekly is still on Monday
Pawan Rawal
@pawanrawal
Apr 14 2016 16:10
Ya team weekly is on Monday, though there is a weekly review tomorrow.
Prashanth Koppula
@koppula
Apr 14 2016 16:11
Ah okay
Ashwin Ramesh
@ashwin95r
Apr 14 2016 16:13
Okay so 8 30 it is
Manish R Jain
@manishrjain
Apr 14 2016 22:00
If it’s too early, we could move them an hour or a couple of hours later.