Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Glen K. Peterson
    @GlenKPeterson
    You also asked "how" of Paguro - Clojure source code is open source and written in Java. I basically forked the collections and added type annotations. I'd made something like Java 8 Streams in Java 7 (again, 'cause I liked how Clojure and Scala did that), so I combined the two and that's basically Paguro. The only down side is it's effectively Eclipse 1.0. I'd rather have Apache 2.0, so if you copy parts of it that aren't from Clojure, you're free to pick either license.
    No, It sounds like you're working with 3D. I only ever really understood 2D really well. 3D is a little bit above my pay grade. I'm maybe 3 or 4 math courses shy of really being prepared for that. Maybe some day.
    Jeffrey Elrod
    @jfelrod1960
    3D is a challenge ... that's for sure.
    Jeffrey Elrod
    @jfelrod1960
    Glen unable to come tonight. Had to go to the urgent care center yesterday. See you next time.
    Jeffrey Elrod
    @jfelrod1960
    Glen how did your talk go?
    Glen K. Peterson
    @GlenKPeterson
    Hi Jeff. It went very well. Thank you for asking. I had prepared pretty well, every time I looked up, everyone was bright-eyed and paying attention, lots of questions... It was a lot of information, but they held up well. :-) I appreciated the very knowledgeable and encouraging audience. I appreciate your support as well! I always learn something preparing for a talk like this, which is a nice by-product.
    Jeffrey Elrod
    @jfelrod1960
    That's great Glen! I wish I was there.
    Jeffrey Elrod
    @jfelrod1960
    Glen I see an invitation to Stomping Grounds next Monday. I thought it had closed?
    Glen K. Peterson
    @GlenKPeterson
    Closed for July 4th weekend only. Thanks for checking. Maybe I'll see you there?!
    Jeffrey Elrod
    @jfelrod1960
    Yes I will be there ... Thanks! :)
    Glen K. Peterson
    @GlenKPeterson
    :-)
    Jeffrey Elrod
    @jfelrod1960
    Glen if I want to get the new changes you are making I should get the 5-22-16 branch?
    Glen K. Peterson
    @GlenKPeterson
    Hey Jeff! Good to see you again.
    Yes, this branch is release-candidate status: https://github.com/GlenKPeterson/Paguro/tree/2016-05-22_RRB-Tree
    To do: 1. improve test coverage of Cowry (copy-on-write-array) methods for completeness. I think that's pretty minor.
    I had some wish-list items like an optimized ListIterator for the RRBTree, but that's going to be hard and I hardly ever use ListIterator (I just use Iterator). I might add "OrNone" versions of the Union types: OneOf3 and OneOf4. There's already a OneOf2OrNone.
    I'm using it more and more in Kotlin and there's some namespace collision between the names I used and the names of the Kotlin standard libraries. So I'll probably have to rename a few things for maximum smoothness in Kotlin. I'm still pretty unsure about every aspect of that, so don't wait.
    Glen K. Peterson
    @GlenKPeterson
    I guess the one thing I haven't completed are upgrade instructions. If you used the 2.x releases, there's a lot of renaming and deprecation. I guess I have to make my Kotlin decisions before I can document all of that. So probably another couple weeks before that becomes an actual release. Otherwise, I think the functionality is arguably complete and correct.
    Jeffrey Elrod
    @jfelrod1960
    Good to hear from you too! I'm just playing around with the library.
    Glen K. Peterson
    @GlenKPeterson
    Yes, definitely use 3.0.12-BETA.
    Jeffrey Elrod
    @jfelrod1960
    I should see you guys soon. Just dealing with life. I will use that version. Thanks for your work.
    Glen K. Peterson
    @GlenKPeterson
    You have to pull and build it yourself. Should just require Java 8 and Maven. I can't upload new versions to Sonatype (maven central) for some reason. I think the change is on their end, but don't know.
    Thanks for taking an interest!
    Alan Dipert
    @alandipert
    hey Glen, thanks for your interest in the problem we had
    the problem was that i have a sealed class Event with two subclasses, and an RrbTree declared like this: var script: RrbTree<Event>
    the "cow" stuff fell over rebalancing the tree or something, copying arrays, until i declared like var script: RrbTree<out Event>instaed
    i'm thinking somewhere underneath it had created an array of the type of one of the subclasses, and then tried to put the other kind of subclass in it or something and jvm puked
    Glen K. Peterson
    @GlenKPeterson
    Thank you for this. CowArray stands for Copy On Write Array utilities. There's so much of this going on that I made sort of primitive functions that maximize the use of System.arrayCopy. On the JVM arrays are still reified so the type of the array is still available at runtime. I have certainly seen this type of error before, but got to a point where I couldn't find them any more.
    Glen K. Peterson
    @GlenKPeterson
    The problem with the Cowry stuff is that the type checker curls up and dies on it, so it's really easy to code a type error bug. IDK. I sort of wanted to keep using types where practical.
    You didn't catch an exception or line number did you?
    I mean a stack trace?
    If you could post that, it would be awesome.
    It's possible that in Kotlin I could set the variance on RrbTree so that when you say RrbTree<Event> it knows it's going to be an "out" param. Or it could just be a bug in my code.
    So, my other litmus test is to ask if you see the same error using Kotlin's listOf() and stuff? If that does the right thing, than my RrBTree should be fixable to behave like that.
    Anyway, thanks for reporting this. Even if I don't hear from you again, I'll try to make a unit test that shows this issue before end-of-year and fix the problem if I can find it. Cheers!
    Alan Dipert
    @alandipert
    @GlenKPeterson sure, i can put together a repro for you
    when we changed the function to return an ArrayList instead, 'out' didn't seem necessary anymore... so i imagine rrbtree is fixable
    Glen K. Peterson
    @GlenKPeterson
    Thank you @alandipert ! That's very helpful. I'll work on reproducing it in a unit test this weekend and fixing it. I want to say, "If ArrayList doesn't have the issue, then it's a bug" but Kotlin uses a special wrapper for ArrayList. Be aware that the fix might require rewriting Paguro in Kotlin - a process that's about 20% complete, but probably won't be done before February at this rate.
    Idea 1: If a PersistentVector is just as good for you, use vec() and return an ImList. That's much better tested. RrbTree is still pretty new.
    Glen K. Peterson
    @GlenKPeterson
    Idea 2: If you stick with an RrbTree, consider returning ImRrbt or MutableRrbt instead of RrbTree. I'm not sure if that's good advice or not.
    Idea 3: Use the mutable tree in your fold, then call .immutable() when done if you want a maybe 5x speed improvement:
    .fold(RrbTree.emptyMutable<Event>()) { events, line -> events.append(parseLine(line)) }.immutable()
    Glen K. Peterson
    @GlenKPeterson
    Idea 4: Again RrbTree is too new to be in Paguro's Xform/Transformation API, but this should work:
        xform(File(logPath).readLines())
                .filter { !it.startsWith("#") }
                .toImList()
    Internally, it captures everything in a mutable vector, then calls .immutable() before returning. File.readlines() builds a list, but there is no list built by .filter(), only by .toImList(). Just a thought.
    Alan Dipert
    @alandipert
    thanks for the ideas Glen, much appreciated
    Glen K. Peterson
    @GlenKPeterson
    I made the following test which does not show the issue. I believe you. I'm going to have to actually pull down your code and delete stuff until it works, then add stuff back until it breaks.
    package org.organicdesign.fp.collections
    
    import junit.framework.TestCase.assertEquals
    import org.junit.Test
    
    class RrbTreeKotlinTest {
        sealed class Event {
            enum class HTTPEventType { REQ, REQ_HOME, REQ_SINF, REQ_TOK }
            data class HTTPEvent(val type: HTTPEventType) : Event()
    
            enum class WSEventType { WS_OPEN, WS_RECV, WS_RECV_INIT, WS_SEND }
            data class WSEvent(val type: WSEventType) : Event()
        }
    
        fun parseLine(i: Int): Event =
                if (i % 2 == 1) {
                    Event.HTTPEvent(Event.HTTPEventType.values()[i%4])
                } else {
                    Event.WSEvent(Event.WSEventType.values()[i%4])
                }
    
        val numItems:Int = 5000
    
        @Test
        fun testTypeArrays() {
            val test:kotlin.collections.List<Int> = (1 .. numItems).toList()
    //        println("test=$test")
            val test2: RrbTree<Event> = test.asSequence()
                    .fold(RrbTree.empty<Event>(),
                          { events, line -> events.append(parseLine(line)) })
    //        println("test2=$test2")
            assertEquals(numItems, test2.size)
        }
    }
    Anyway, I tried, but it doesn't look like it's happening today.
    Glen K. Peterson
    @GlenKPeterson

    @alandipert I'm sorry, but I can't reproduce this issue. I added the following test to the exact commit of your project that you referenced:

        @Test fun testReadEventLog() {
            val events = readEventLog("src/test/resources/testFile")
    //        println("events=$events")
            assertEquals(2400, events.size)
            val event0 = events[0]
            assertEquals(HTTPEvent::class, event0::class)
    
            val event1 = events[1800]
            assertEquals(WSEvent::class, event1::class)
        }

    And removed "out" from the type signature in your project, and it still worked. Here's the test file I used: https://pastebin.com/txepakng

    I'm going to work on something else unless I hear from you. Nice to hear from you again. Happy new year!

    Alan Dipert
    @alandipert
    @GlenKPeterson no problem, i will take a moment this evening to try to get you a decent minimal repro. it might have been related to the particular input file we used, which i only have locally
    Jeffrey Elrod
    @jfelrod1960
    Hi Glen 😊. When is the next ACM meet?