by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jul 03 22:07

    jgaskins on experimental-bolt-conversion

    Remove retries from the connect… Use lock-free atomic mutation i… Increase max idle pool size Th… and 1 more (compare)

  • Jun 05 02:56

    jgaskins on experimental-bolt-conversion

    Handle busted exec_cast queries… (compare)

  • Jun 03 03:01

    jgaskins on experimental-bolt-conversion

    Clarify specs, add Bolt seriali… (compare)

  • May 06 23:51

    jgaskins on experimental-bolt-conversion

    Make Driver a superclass of bot… (compare)

  • Apr 29 23:06

    jgaskins on experimental-bolt-conversion

    Add experimental to_bolt_params… (compare)

  • Apr 17 21:48
    jgaskins closed #7
  • Apr 17 01:01
    anamba commented #7
  • Apr 16 23:15
    jgaskins commented #7
  • Apr 16 23:13

    jgaskins on master

    Update README with cluster supp… (compare)

  • Apr 16 22:52

    jgaskins on v0.6.0

    (compare)

  • Apr 16 22:52

    jgaskins on master

    Version 0.6.0 (compare)

  • Apr 16 22:51

    jgaskins on master

    Add initial support for clusters Add SNI for SSL connection Merge branch 'master' into add-… and 20 more (compare)

  • Apr 16 22:48

    jgaskins on add-cluster-support

    Update Cluster for Crystal 0.34 Don't spin up an initial connec… (compare)

  • Apr 14 04:14

    jgaskins on add-cluster-support

    Fix StreamingResultSet Support Crystal 0.34 (compare)

  • Apr 08 12:47
    jgaskins commented #7
  • Apr 08 08:28
    anamba commented #7
  • Apr 08 08:26
    anamba opened #7
  • Mar 20 01:30

    jgaskins on add-cluster-support

    Fix compatibility with Crystal … Enforce string key in failure r… (compare)

  • Feb 27 01:31

    jgaskins on add-cluster-support

    Make cluster refresh more resil… (compare)

  • Jan 31 04:09

    jgaskins on add-cluster-support

    Add neo4j as a cluster URL sche… (compare)

Jamie Gaskins
@jgaskins
Because each of these specs pass:
      {
        0_i8,
        -1_i8,
        1_i8,
        Int8::MAX,
        Int8::MIN,

        0_i16,
        -1_i16,
        1_i16,
        Int8::MIN.to_i16 - 1,
        Int8::MAX.to_i16 + 1,
        Int16::MIN,
        Int16::MAX,

        0_i32,
        -1_i32,
        1_i32,
        Int16::MIN.to_i32 - 1,
        Int16::MAX.to_i32 + 1,
        Int32::MIN,
        Int32::MAX,

        0__i64,
        -1_i64,
        1__i64,
        Int32::MIN.to_i64 - 1,
        Int32::MAX.to_i64 + 1,
        Int64::MIN,
        Int64::MAX,
      }.each do |int|
        it "serializes #{int}" do
          unpack(pack(int)).should eq int
        end
      end
Unless I've missed something
Either way, I don't know what the implications of just boosting all ints to Int64 but that certainly might help with your overflows.
Wait, didn't Crystal add overflow detection in 0.27?
Jamie Gaskins
@jgaskins

In this version, a couple of arithmetic operators were added thanks to #6890: &+, &-, &. They will perform additions, subtraction and multiplication with wrapping (as in, not overflowing). Some may notice that that is the exact same behaviour as +, -, . In a future version, the regular operators are going to raise on overflow. This will allow you to trust the result of the operations when reaching the limits of the representable range. The ampersand operators might be useful to think of them as applying an & 0xFF bit mask to the result.

Ah, looks like they only aliased the current, non-raising versions so people can migrate their code if they don't want to raise on overflow. https://crystal-lang.org/2018/11/01/crystal-0.27.0-released.html

It was stack overflow detection they added for real.
Jamie Gaskins
@jgaskins
Ugh, looks like Crystal 0.27.2 broke my async_it monkeypatch on Spec … or I'll have to come up with a different approach to run integration specs concurrently. :joy:
Jamie Gaskins
@jgaskins

Hmm, so it looks like Neo4j is sending us the numbers in the smallest type possible:

result = connection.execute "RETURN $int8, $int16, $int32, $int64",
  int8: Int8::MAX,
  int16: Int16::MAX,
  int32: Int32::MAX,
  int64: Int64::MAX

pp result.first
# => [127_i8, 32767_i16, 2147483647, 9223372036854775807_i64]

We're deserializing it in whatever integer type Neo4j tells us it is. :-\

@anamba I kinda wanna just upcast everything to Int64 to avoid the int-overflow issue you're seeing, but also I think that might break things like Neo4j.map_node(some_int_value: Int16).
Not sure of a way to navigate this yet.
Do you have any thoughts?
Aaron Namba
@anamba
eh, i think it's ok. it's more of a crystal gotcha than a neo4j.cr gotcha i think
but the issue of not being able to save an int value in the range of -17 to -32 is pretty strange
Jamie Gaskins
@jgaskins
Okay, cool. I'll still give it a look and see what the implications are. I normally use map_node and map_relationship to extract nodes and rels to structs and use them as value objects.
Yeah, that part is weird. I wonder if that's also an issue in MsgPack
Lemme run a quick test.
Jamie Gaskins
@jgaskins
Oooh, it looks like it was meant to save a byte if it could. If it's in a certain range, the protocol can cram the byte marker and the value into the same byte, but I didn't decode it properly. I'm just gonna go ahead and encode the byte marker and the value in separate bytes for now until I understand how it works.
@anamba Fixed and pushed v0.2.3
Aaron Namba
@anamba
:confetti_ball:
Jamie Gaskins
@jgaskins
Sorry about that, man
Aaron Namba
@anamba
you have no idea :joy:
that was a WEIRD one
this binary protocol stuff makes for some pretty strange corner cases. not in the corners i usually look for
Jamie Gaskins
@jgaskins
lol yeah, the JSON version is super easy because everything's a Float64. it's been a long time since I've worked in a language that distinguished between numbers of different sizes and even longer since I had to parse binary protocols in them. :joy:
Aaron Namba
@anamba
but hey overall, don't feel too bad about it or anything. i knew what i was signing up for when i built real apps on a < 1.0 language, talking to a somewhat niche db over a version 0.1 (at the time) db driver, and rolling my own ORM :smile:
Jamie Gaskins
@jgaskins
lol
Aaron Namba
@anamba
having a changelog or release notes for neo4j.cr would be handy :) right now i look at the commits, which is not so terrible right now since there aren't many
checked it just now to fix my travis build. knew that ancient version of neo4j would be a problem eventually
i copied your .travis.yml stuff, made a few small improvements:
before_install:
  - curl -sSL https://neo4j.com/artifact.php?name=neo4j-community-3.5.2-unix.tar.gz | tar x
  - cd neo4j-community-3.5*
  - bin/neo4j start
  - false ; while [ $? -ne 0 ]; do sleep 1 ; curl -sI http://localhost:7474 | head -1 | grep 200 ; done
Jamie Gaskins
@jgaskins
I've been meaning to add a changelog. Just haven't gotten there yet. :-)
Jamie Gaskins
@jgaskins
@anamba I've been experimenting with bumping the supported Bolt version to v3 (Neo4j 3.5+) and it looks like a bunch of things have changed. The v2 bump caused some issues for you, so I wanted to make sure you were aware. If I end up merging this branch, I'll bump the minor version and let you know.
Aaron Namba
@anamba
@jgaskins I've actually encountered the occasional Neo4j 3.5 specific issue, so being locked into that might not be ideal. But time marches on and it's fun to support new things, so go for it. But thanks for the heads up.
Jamie Gaskins
@jgaskins
:+1:
Aaron Namba
@anamba
@jgaskins so i got another really weird one for ya. hope you can make some sense of it because i sure can't. this happens constantly. i was holding off for a bit, hoping 0.31.0 would fix it somehow, but nope.
IO::Error

Can't write to ARGF

    /usr/share/crystal/src/io/argf.cr in write at line 58:5

          nil
        end
      end
      def write(slice : Bytes) : NoReturn
        raise IO::Error.new "Can't write to ARGF"
      end
      def path
        @path || @argv.first? || "-"
      end

/usr/share/crystal/src/io/buffered.cr in run at line 223:51
/home/app/myapp/lib/neo4j/src/neo4j/bolt/connection.cr in ??? at line 522:15
/home/app/myapp/lib/neo4j/src/neo4j/bolt/connection.cr in execute at line 475:9
according to my shard.lock this is neo4j.cr 0.5.5
ah i should have said, it happens constantly in production (as in, i have thousands of events in Sentry), but i have not yet been able to reproduce the issue locally. :-D which is why i am hoping you have some idea what this means.
Aaron Namba
@anamba
also, i am trying out -Dpreview_mt in 0.31.0 and am getting random errors like this:
Invalid memory access (signal 11) at address 0x39
[0x56137e652cb6] *CallStack::print_backtrace:Int32 +118
[0x56137e635439] __crystal_sigfault_handler +217
[0x7fec1f0c9890] ???
[0x56137e707f01] *Zlib::Writer +1
[0x56137e70877a] *Zlib::Writer#flush:Nil +10
[0x56137e7700b0] *Neo4j::Bolt::Connection#send<Neo4j::Bolt::Connection::Commands, String, Hash(String, Neo4j::Value)>:(IO+ | Int32 | Nil) +2576
[0x56137e76edc2] *Neo4j::Bolt::Connection#run<String, Hash(String, Neo4j::Value), Int32>:(Neo4j::Ignored | Neo4j::Success) +82
[0x56137e76ed53] *Neo4j::Bolt::Connection#run<String, Hash(String, Neo4j::Value)>:(Neo4j::Ignored | Neo4j::Success) +51
[0x56137e76ea18] *Neo4j::Bolt::Connection#execute<String, Hash(String, Neo4j::Value)>:Neo4j::Result +88
[0x56137e76e9af] *Neo4j::Bolt::Connection#execute<String>:Neo4j::Result +47
Aaron Namba
@anamba
i'm starting to realize there could be a number of things in the stdlib that are not thread safe yet
Jamie Gaskins
@jgaskins
@anamba I'm so sorry, I haven't been paying any attention to Gitter lately. Feel free to post bugs on the issue tracker when they come up!
Jamie Gaskins
@jgaskins
I haven't seen that bug with this shard, but I see ARGF come up a lot when working with subclasses of IO — might be the type system getting confused somehow, but I honestly haven't seen that at all.
Regarding -Dpreview_mt, I'm not sure what that could be, either. Bolt doesn't use compression, so I'm not sure why Zlib is in the stack trace at all.
Jamie Gaskins
@jgaskins
The only issues I've had with multithreading with Neo4j was in using the pool shard — it doesn't appear to be threadsafe yet. :-\ I did try pulling the connection pool out of crystal-db and it worked just fine for one app (I did have to remove the callbacks it uses, but you could also probably monkeypatch those methods onto Neo4j::Bolt::Connection), so if you need thread safety, that might be an option.