You also asked "how" of Paguro - Clojure source code is open source and written in Java. I basically forked the collections and added type annotations. I'd made something like Java 8 Streams in Java 7 (again, 'cause I liked how Clojure and Scala did that), so I combined the two and that's basically Paguro. The only down side is it's effectively Eclipse 1.0. I'd rather have Apache 2.0, so if you copy parts of it that aren't from Clojure, you're free to pick either license.
No, It sounds like you're working with 3D. I only ever really understood 2D really well. 3D is a little bit above my pay grade. I'm maybe 3 or 4 math courses shy of really being prepared for that. Maybe some day.
Hi Jeff. It went very well. Thank you for asking. I had prepared pretty well, every time I looked up, everyone was bright-eyed and paying attention, lots of questions... It was a lot of information, but they held up well. :-) I appreciated the very knowledgeable and encouraging audience. I appreciate your support as well! I always learn something preparing for a talk like this, which is a nice by-product.
Yes, this branch is release-candidate status: https://github.com/GlenKPeterson/Paguro/tree/2016-05-22_RRB-Tree
To do: 1. improve test coverage of Cowry (copy-on-write-array) methods for completeness. I think that's pretty minor.
I had some wish-list items like an optimized ListIterator for the RRBTree, but that's going to be hard and I hardly ever use ListIterator (I just use Iterator). I might add "OrNone" versions of the Union types: OneOf3 and OneOf4. There's already a OneOf2OrNone.
I'm using it more and more in Kotlin and there's some namespace collision between the names I used and the names of the Kotlin standard libraries. So I'll probably have to rename a few things for maximum smoothness in Kotlin. I'm still pretty unsure about every aspect of that, so don't wait.
I guess the one thing I haven't completed are upgrade instructions. If you used the 2.x releases, there's a lot of renaming and deprecation. I guess I have to make my Kotlin decisions before I can document all of that. So probably another couple weeks before that becomes an actual release. Otherwise, I think the functionality is arguably complete and correct.
Thanks for taking an interest!
the problem was that i have a sealed class Event with two subclasses, and an RrbTree declared like this: var script: RrbTree<Event>
the "cow" stuff fell over rebalancing the tree or something, copying arrays, until i declared like
var script: RrbTree<out Event>instaed
i'm thinking somewhere underneath it had created an array of the type of one of the subclasses, and then tried to put the other kind of subclass in it or something and jvm puked
Thank you for this. CowArray stands for Copy On Write Array utilities. There's so much of this going on that I made sort of primitive functions that maximize the use of System.arrayCopy. On the JVM arrays are still reified so the type of the array is still available at runtime. I have certainly seen this type of error before, but got to a point where I couldn't find them any more.
You didn't catch an exception or line number did you?
I mean a stack trace?
If you could post that, it would be awesome.
It's possible that in Kotlin I could set the variance on RrbTree so that when you say RrbTree<Event> it knows it's going to be an "out" param. Or it could just be a bug in my code.
So, my other litmus test is to ask if you see the same error using Kotlin's listOf() and stuff? If that does the right thing, than my RrBTree should be fixable to behave like that.
Anyway, thanks for reporting this. Even if I don't hear from you again, I'll try to make a unit test that shows this issue before end-of-year and fix the problem if I can find it. Cheers!
for starters, this is the offending code. broke when called without use of 'out' https://github.com/rstudio/proxyrec/blob/4e1efddcd1b773ada57068430af38f14b1341f09/player/src/main/kotlin/com.rstudio/Main.kt#L59-L66
when we changed the function to return an ArrayList instead, 'out' didn't seem necessary anymore... so i imagine rrbtree is fixable
Thank you @alandipert ! That's very helpful. I'll work on reproducing it in a unit test this weekend and fixing it. I want to say, "If ArrayList doesn't have the issue, then it's a bug" but Kotlin uses a special wrapper for ArrayList. Be aware that the fix might require rewriting Paguro in Kotlin - a process that's about 20% complete, but probably won't be done before February at this rate.
Idea 1: If a PersistentVector is just as good for you, use vec() and return an ImList. That's much better tested. RrbTree is still pretty new.
Idea 3: Use the mutable tree in your fold, then call .immutable() when done if you want a maybe 5x speed improvement:
.fold(RrbTree.emptyMutable<Event>()) { events, line ->
events.append(parseLine(line))
}.immutable()
Internally, it captures everything in a mutable vector, then calls .immutable() before returning. File.readlines() builds a list, but there is no list built by .filter(), only by .toImList(). Just a thought.
I made the following test which does not show the issue. I believe you. I'm going to have to actually pull down your code and delete stuff until it works, then add stuff back until it breaks.
package org.organicdesign.fp.collections
import junit.framework.TestCase.assertEquals
import org.junit.Test
class RrbTreeKotlinTest {
sealed class Event {
enum class HTTPEventType { REQ, REQ_HOME, REQ_SINF, REQ_TOK }
data class HTTPEvent(val type: HTTPEventType) : Event()
enum class WSEventType { WS_OPEN, WS_RECV, WS_RECV_INIT, WS_SEND }
data class WSEvent(val type: WSEventType) : Event()
}
fun parseLine(i: Int): Event =
if (i % 2 == 1) {
Event.HTTPEvent(Event.HTTPEventType.values()[i%4])
} else {
Event.WSEvent(Event.WSEventType.values()[i%4])
}
val numItems:Int = 5000
@Test
fun testTypeArrays() {
val test:kotlin.collections.List<Int> = (1 .. numItems).toList()
// println("test=$test")
val test2: RrbTree<Event> = test.asSequence()
.fold(RrbTree.empty<Event>(),
{ events, line -> events.append(parseLine(line)) })
// println("test2=$test2")
assertEquals(numItems, test2.size)
}
}
Anyway, I tried, but it doesn't look like it's happening today.
@alandipert I'm sorry, but I can't reproduce this issue. I added the following test to the exact commit of your project that you referenced:
@Test fun testReadEventLog() {
val events = readEventLog("src/test/resources/testFile")
// println("events=$events")
assertEquals(2400, events.size)
val event0 = events[0]
assertEquals(HTTPEvent::class, event0::class)
val event1 = events[1800]
assertEquals(WSEvent::class, event1::class)
}And removed "out" from the type signature in your project, and it still worked. Here's the test file I used: https://pastebin.com/txepakng
I'm going to work on something else unless I hear from you. Nice to hear from you again. Happy new year!