Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jul 10 2015 16:26

    kmario23 on master

    remove stemmers update report with KL divergenc… (compare)

  • Jul 10 2015 15:09

    kmario23 on master

    add crude code for KLD add code for KL-divergence/Cros… (compare)

  • Jul 06 2015 07:13

    kmario23 on ex7-mario

    (compare)

  • Jul 05 2015 10:42

    kmario23 on master

    remove ex8.tex (compare)

  • Jul 05 2015 09:49

    kmario23 on master

    add data for ex9 (compare)

  • Jul 05 2015 09:39

    kmario23 on master

    add report for ex7 add code for entropy calculation (compare)

  • Jul 03 2015 16:16

    bryandeng on master

    Fix typo in ex8 (compare)

  • Jul 03 2015 15:35

    bryandeng on master

    Add proof for Bonus 1 (compare)

  • Jul 03 2015 15:06

    bryandeng on ex8-boyuan

    (compare)

  • Jul 03 2015 15:05

    bryandeng on master

    Add report for ex8 (compare)

  • Jul 03 2015 14:57

    bryandeng on ex8-boyuan

    (compare)

  • Jul 03 2015 12:25

    kmario23 on ex8-mario

    add code for entropy (compare)

  • Jun 26 2015 21:41

    kmario23 on ex7-mario

    final tex code for review; code… (compare)

  • Jun 26 2015 21:28

    kmario23 on ex7-mario

    final tex code for review (compare)

  • Jun 26 2015 20:59

    steffervescency on master

    Ex07 4a (compare)

  • Jun 26 2015 15:52

    bryandeng on master

    Return real probability in "sen… (compare)

  • Jun 26 2015 15:13

    kmario23 on ex7-mario

    tex file for report ex7 (compare)

  • Jun 26 2015 14:56

    bryandeng on ex7-boyuan

    (compare)

  • Jun 26 2015 14:56

    bryandeng on master

    Implement function "sentence_sc… Implement function "rank_valid_… Implement function "sentence_sc… and 3 more (compare)

  • Jun 26 2015 14:56
    bryandeng closed #13
steffervescency
@steffervescency
but Dkl(tom sawyer 1 || tom sawyer german) = 4.481277461789773
I guess I don't know which sort of values we should be expecting, but somehow I thought the last one would be higher
steffervescency
@steffervescency
you take away probability mass from existing tokens - the idea is that you want to still sum up to 1 in the end
hahaha yeah :smile:
Ghost
@ghost~554f6a5315522ed4b3e02e44
:) And we can also see that, in this case, it(Dkl) depends on the epsilon value. I'm also not sure whether 10^(-4) is the value that we should use.
steffervescency
@steffervescency
yeah hmm, eps = 10**(-9) gives us 17.399193004643458 for Dkl(tom sawyer 1 || tom sawyer german)
steffervescency
@steffervescency
i'm not sure what the right value is - i think we should just write that in the report and talk about it in the tutorial (I'll go this time then! haha)
Ghost
@ghost~554f6a5315522ed4b3e02e44
Sure!! :+1:
And, when I use 10**(-9) for Dkl(tom sawyer 1 || tom sawyer german), I get 17.093741716932275
Ghost
@ghost~554f6a5315522ed4b3e02e44
with epsilon = 10**(-4), I get the following :
Dkl(huck finn || moby dick) = 1.8409885468327254
Dkl(tom sawyer 1 || tom sawyer 2) = 0.33076453519322735
Dkl(tom sawyer 1 || tom sawyer german) = 4.398161493734904
I can't guess why we have a slight variance for last case :worried:
Ghost
@ghost~554f6a5315522ed4b3e02e44

And, for 1c) What can we write? May be something like,
KL-divergence is not symmetric. Because,
Dkl(tom sawyer 1 || tom sawyer german) : 4.398161493734893
Dkl(tom sawyer german || tom sawyer 1) : 3.2544862459626716

Also, Cross Entropy is relatively high between languages

steffervescency
@steffervescency
i would say what KL divergence means (# of wasted bits using the wrong encoding) + some comparison between two divergences maybe?
can you do the merge and submit the assignment after? i don't think I will be around later
Ghost
@ghost~554f6a5315522ed4b3e02e44
Sure! I'm done with the report work.
And Thanks for the huffman code code :)
Ghost
@ghost~554f6a5315522ed4b3e02e44
huffman.py gives 'math domain error' when run from commandline but works perfectly fine when run in spyder. This is kinda weird.
B: how's it going ?
steffervescency
@steffervescency
thanks for the submission, it looks great :D
that's odd, with which version of python?
Ghost
@ghost~554f6a5315522ed4b3e02e44
:+1:
Yeah, I later realized that when run from the command line, only Python2.x throws the error while it's perfectly fine in Python3.x
Ghost
@ghost~554f6a5315522ed4b3e02e44
But when we run it using Spyder, it's perfectly fine. So I think, the IDE uses the relevant interpreter based on the code analysis.
Ghost
@ghost~554f6a5315522ed4b3e02e44
I came to your place Bryan. Aren't you in SB?
Ghost
@ghost~554f6a5315522ed4b3e02e44
I haven't progressed much learning this. But the contents are worth checking it out.
steffervescency
@steffervescency
already have been working through it :D let me know if you want to go over any of it together after exams!
Ghost
@ghost~554f6a5315522ed4b3e02e44
Thanks Stef! I'd be very glad to join :)
Ghost
@ghost~554f6a5315522ed4b3e02e44
Ghost
@ghost~554f6a5315522ed4b3e02e44
prize money seems pretty attractive:
steffervescency
@steffervescency
that's good to know, thanks :D were you in the lecture this morning?
also, would you guys mind skipping this assignment? i'd rather spend more time on studying than debugging code, and we don't need the assignment to qualify for the exam at this point
Ghost
@ghost~554f6a5315522ed4b3e02e44
I'm thinking on the same lines too.
Today's lecture was "5 minute summaries" of all the chapters.
steffervescency
@steffervescency
yeah, I was at the lecture, but thanks :)
steffervescency
@steffervescency
isn't that the same as the definition in the slides?
steffervescency
@steffervescency
1) I think that's right with entropy
steffervescency
@steffervescency
entropy for uniform distribution is just log_2(|V|) right?
steffervescency
@steffervescency
what did you guys think for: 16. You are writing up your thesis. Which distribution is suitable to describe the numbers of typos per page (formula and explanation)?
I'm pretty sure it's the poisson distribution, but I can't remember talking about that in class
steffervescency
@steffervescency
ahh, thank you!
Ghost
@ghost~554f6a5315522ed4b3e02e44
:+1:
steffervescency
@steffervescency
i think it's right!
steffervescency
@steffervescency
tonight is a little difficult for me, but there's a coli review session tomorrow morning from 9:30 if you want to join!
we are meeting in the coli lounge - take a left when you enter the coli building, go down the stairs, and it's the first room on the left after the bathroom
steffervescency
@steffervescency
np! and it's a different flight of stairs, in the opposite direction
steffervescency
@steffervescency
Thanks to both of you for a good semester too :D good luck with the rest of your exams!
Ghost
@ghost~554f6a5315522ed4b3e02e44
:+1: