These are chat archives for jruby-gradle/jruby-gradle-plugin

16th
Aug 2017
Asfand Qazi
@ayqazi
Aug 16 2017 16:54
Hello.
I want to include a Ruby gem in my gradle dependencies, but it comes from somewhere other than rubygems.org . How do I accomplish this? It's a private repo accessible by HTTP URL, and works fine in bundler, and via the rubygems command-line.
Thanks in advance.
R. Tyler Croy
@rtyler
Aug 16 2017 17:02
@ayqazi that would require using the built-in gem proxy unfortunately
Asfand Qazi
@ayqazi
Aug 16 2017 17:03
Those are the docs I was looking for, thanks! Don't know how I missed that, I was looking at that site for a whole day
R. Tyler Croy
@rtyler
Aug 16 2017 17:03
heh, no worries
those docs are on github, so if you have suggestions or ideas for improvement, please make them! :D
Asfand Qazi
@ayqazi
Aug 16 2017 17:04
Sure, I'll keep that in mind
Asfand Qazi
@ayqazi
Aug 16 2017 17:13

So I have this now:

repositories {
    mavenCentral()
    jcenter()
    rubygems('https://rubygems.org')
    rubygems('https://PRIVATETOKEN@gem.fury.io/mycompany')
}

dependencies {
    jrubyJar 'rubygems:colorize:0.7.7'
    jrubyJar 'rubygems:mycompanygem:[0.1.0,1.0.0]'
}

This gives me the error:

  > Could not get resource 'http://localhost:45635/https_PRIVATETOKEN@gem_fury_io_mycompany/caching/maven/releases/rubygems/mycompanygem/0.1.0/mycompanygem-0.1.0.gem'.
     > Could not HEAD 'http://localhost:45635/https_PRIVATETOKEN@gem_fury_io_mycompany/caching/maven/releases/rubygems/mycompanygem/0.1.0/mycompanygem-0.1.0.gem'. Received status code 500 from server: Server Error

Can you see what I'm doing wrong?

R. Tyler Croy
@rtyler
Aug 16 2017 17:14
eeep
can you run that with --stacktrace I wonder if the built-in gem proxy is explodiing
Asfand Qazi
@ayqazi
Aug 16 2017 17:19
Here's a gist with full (sanitized) output: https://gist.github.com/ayqazi/74df54a8d3ff2ea86095aa608ffcf051
R. Tyler Croy
@rtyler
Aug 16 2017 17:20
interesting, @mkristian does that look familiar to you at all?
R. Tyler Croy
@rtyler
Aug 16 2017 17:25
@ayqazi what version?
Asfand Qazi
@ayqazi
Aug 16 2017 17:25
1.4.0
R. Tyler Croy
@rtyler
Aug 16 2017 17:26
hm
Asfand Qazi
@ayqazi
Aug 16 2017 17:28
Same thing with version 1.5.0 of the plugin
R. Tyler Croy
@rtyler
Aug 16 2017 17:28
yeah, there weren't really any substantial changes in that support between the two
I understand what's wrong in the stacktrace, but I don't understand why
Asfand Qazi
@ayqazi
Aug 16 2017 17:34
Is it because I've got an '@' in the URL?
R. Tyler Croy
@rtyler
Aug 16 2017 17:34
perhaps
does the rubygems server require basic auth?
Asfand Qazi
@ayqazi
Aug 16 2017 17:35
It does not, but our private repo does
R. Tyler Croy
@rtyler
Aug 16 2017 17:35
that's what I'm referring to :)
Asfand Qazi
@ayqazi
Aug 16 2017 17:35
Oh sorry. Yes, our private repo uses basic auth :) Well it's a gemfury-ism really
Well it looks like I'll have to manually download the gem and add it to the jar. Not too inelegant all things considered. My Javafu is not good enough yet for me to actually try to fix this issue, sorry.
R. Tyler Croy
@rtyler
Aug 16 2017 17:37
I understand :)
@ayqazi could you file an issue for supporting basic auth for the built-in gem proxy
I'm not sure if it's a documentation issue or whether the underlying rubygems-servlets code simply doesn't support it
Asfand Qazi
@ayqazi
Aug 16 2017 17:38
Sure.
Done. I appreciate your help :)
R. Tyler Croy
@rtyler
Aug 16 2017 17:42
I'm not sure what gemfury is
but if that's something you have an easy reproduction/usecase for, that helps too
Asfand Qazi
@ayqazi
Aug 16 2017 17:42
They let you host your own private Rubygems repositories. Edit: I've updated the issue with some steps to allow reproducing the issue using GemFury
R. Tyler Croy
@rtyler
Aug 16 2017 17:46
thanks
Blane Dabney
@raelik
Aug 16 2017 17:57
@ayqazi basic auth works with the gem proxy, has for a while
I submitted the original bug report on that last year I think
R. Tyler Croy
@rtyler
Aug 16 2017 17:58
heh
Blane Dabney
@raelik
Aug 16 2017 17:58
it's technically not a problem in jruby-gradle, it was further upstream
you just need to put the username and password in the rubygems() url
OH, I see... nevermind.
I probably should have looked more carefully at that.
it's because of the token-based auth, instead of the standard username:password format
rtyler @rtyler looks at the gemfury docs
R. Tyler Croy
@rtyler
Aug 16 2017 18:05
how does this token@foo even work
o_O
Blane Dabney
@raelik
Aug 16 2017 18:05
oh wow, that was almost 2 years ago, the authentication patch for sonatype to support basic auth
R. Tyler Croy
@rtyler
Aug 16 2017 18:06
time flies when you're building codes
Blane Dabney
@raelik
Aug 16 2017 18:06
no kidding
R. Tyler Croy
@rtyler
Aug 16 2017 18:14
@raelik do you happen t o know if this token@foo is even using the http basic auth format underneath? I've never seen it before
or perhaps this is relying on some functionality in rubygems itself to determine what to do with that token when it makes the HTTP request
Blane Dabney
@raelik
Aug 16 2017 18:14
It isn't standard.
but technically, the username:password style isn't either
R. Tyler Croy
@rtyler
Aug 16 2017 18:18
I think this might be a rubygems thing
their npm example doesn't use this format, for example
Blane Dabney
@raelik
Aug 16 2017 18:18
I've never seen that before, we're using geminabox, and it doesn't do that
just plain username:password
technically, the bit before the @ can be anything.
It isn't actually specified in the HTTP RFC
basically everyone settled on using username:password for Basic auth though
rtyler @rtyler nods
R. Tyler Croy
@rtyler
Aug 16 2017 18:20
@ayqazi I wonder if you are able to use that URL format with curl to fetch a gem or manifest file from gemfury?
Blane Dabney
@raelik
Aug 16 2017 18:20
BTW, here's the original patch that mkristian put in to get sonatype's nexus-ruby-tools working with basic auth: sonatype/nexus-public@11fb0e3
well, his patch is included in that mega-patch from their private repo
rtyler @rtyler looks
Blane Dabney
@raelik
Aug 16 2017 18:22
So, looking at that, I don't think it being not in username:password format actually matters
because it doesn't seem to care. It just grabs the entire 'userinfo' section of the URL and sticks it in the Authorization header.
with 'Basic' in front to tell it to do basic auth
R. Tyler Croy
@rtyler
Aug 16 2017 18:24
@raelik did you see @ayqazi's traceback above?
Blane Dabney
@raelik
Aug 16 2017 18:25
yup, I'm digging into the sonatype code right now
pretty sure it's RubgemsFile#get() somehow returning an Exception instead of throwing it
Oh, it's supposed to do that
R. Tyler Croy
@rtyler
Aug 16 2017 18:27
O_O
Blane Dabney
@raelik
Aug 16 2017 18:28
get() returns whatever the payload is
and the payload can be an exception
R. Tyler Croy
@rtyler
Aug 16 2017 18:28
but #handle doesn't seem to be...handling that :)
Blane Dabney
@raelik
Aug 16 2017 18:28
SimpleStorage#getInputStream is just blindly casting that to a StreamLocation without bothering to see if it's an exception first.
Wait... that's weird.
So getInputStream DOES check to see if the payload is an exception first, and throws it
how is that traceback even possible?
R. Tyler Croy
@rtyler
Aug 16 2017 18:32
aliens.jpg
that is where it's getting thrown, because it's trying to cast an IOException as a StreamLocation.
look at line 116
that hasException() checks if the state is ERROR, which gets set when the payload is an exception
R. Tyler Croy
@rtyler
Aug 16 2017 18:33
hasException() isn't a built-in type
Blane Dabney
@raelik
Aug 16 2017 18:33
it's part of RubygemsFile
R. Tyler Croy
@rtyler
Aug 16 2017 18:34
how confident are you that state is being set properly
perhaps it isn't
rtyler @rtyler clones this repo for some grep on grep action
and also a few lines down in setException
maybe set() is getting called on an Exception by accident somewhere else.
that's what I'd look for.
Blane Dabney
@raelik
Aug 16 2017 18:41
I think I know where it's happening.
Hmmm I don't see how though
Oh wait, no.
I think that's supposed to be in an else { }
yes, that's exactly what it is.
R. Tyler Croy
@rtyler
Aug 16 2017 18:45
hot dog, you found a bug
Blane Dabney
@raelik
Aug 16 2017 18:45
because that traceback happened when it tried to download the .gem file.
which is a gzip file.
booya
of course, that means that there was an exception when it tried to get the gem file :P
that's kinda odd though
I would have expected the following file.set() call to throw an exception
yeah... not so sure about this being the actual source of the bug now.
R. Tyler Croy
@rtyler
Aug 16 2017 18:59
I haven't seen @mkristian in a while, so not sure if we can summon him
rtyler @rtyler tries his best spells
raelik @raelik casts the bones.
Blane Dabney
@raelik
Aug 16 2017 19:06
Your chances of success are slim. It may require a blood sacrifice.
R. Tyler Croy
@rtyler
Aug 16 2017 19:06
heh
well, if he's still in Germany, he's probably offline for the day anwyays
Blane Dabney
@raelik
Aug 16 2017 19:07
Indeed
R. Tyler Croy
@rtyler
Aug 16 2017 19:07
so I don't understand how this issue would only be manifest for @ayqazi
IIRC we have some integration tests around using the built-in proxy
but against rubygems.org
Blane Dabney
@raelik
Aug 16 2017 19:07
I'm sure that somewhere in there it has to do with his non-"standard" userinfo format
some assumption is probably being made somewhere.
R. Tyler Croy
@rtyler
Aug 16 2017 19:08
hm
Blane Dabney
@raelik
Aug 16 2017 19:08
actually, you know what.
I don't think it is.
because there are other files being retrieved successfully from that url
more likely that it has something to do with gemfury
I'm really just trying to figure out how this RubygemsFile object has gotten itself into a non-ERROR state with an Exception in the payload
since that's the only way that traceback could have happened.
honestly, even that example I found shouldn't have been able to do that
maybe this is a race condition??
R. Tyler Croy
@rtyler
Aug 16 2017 19:12
I wonder if perhaps it's setting some content headers which are unecpected for the payload
Asfand Qazi
@ayqazi
Aug 16 2017 19:12
It's happened every single time, even when I only get one gem and only have one rubygems dependency line.
Blane Dabney
@raelik
Aug 16 2017 19:13
setException calls set() first to set the payload as the exception, and then it sets the ERROR state
Asfand Qazi
@ayqazi
Aug 16 2017 19:13
(Comment regarding the race condition h
Blane Dabney
@raelik
Aug 16 2017 19:13
if that is being done in another thread, then it's possible that the state is being set too late.
I didn't think there was any multithreading happening in this particular bit of code though
@ayqazi Yeah, I would expect it to be somewhat transient if it was that basic of a race condition.
Blane Dabney
@raelik
Aug 16 2017 19:42
so it definitely looks like there's a bug with that lack of an else { }, but I don't think that has anything to do with this particular issue.