Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Dave Wichers
    @davewichers
    Thanks for the tip! When I try just scanResult.getAllResources().findDuplicatePaths(), I get lots of irrelevant stuff like flagging: dozens of files like: META-INF/LICENSE. I only care about conflicts with package names (i.e., code). Is there a way to be more specific to only look at that? And of course, to also include what is built into Java itself as well. I'm going to try your above code snippet now to see if that's better.
    Luke Hutchison
    @lukehutch

    If you use .acceptPathsNonRecursive("META-INF/services"), then it should not be finding META-INF/LICENSE, since that file is not in META-INF/services/. Are you calling .acceptPathsNonRecursive("META-INF/services")? If you are, please file a bug in GitHub, with a small reproducer project so that I can fix it.

    As far as I know, anything in META-INF/services should have the package and class name of the service as the file name, so you shouldn't have any files in that directory that are not fully-qualified service names as their filename.

    If you do need to do some filtering of a ResourceList though, you can pass a filtering lambda to ResourceList#filter(...), which produces a new ResourceList in much the same way as Stream#filter(...).

    More on .acceptPaths -- if you don't list any paths to be accepted (as opposed to being rejected), then all paths are accepted. This is why I suspect you missed that call.
    Dave Wichers
    @davewichers
    OK - Using your code block, I was able to get all the service definitions from only the META-INF/services/ folder. So that takes care of all the custom libraries declared for my app. It also found the libraries brought in my tomcat, and the ones from the JDK (e.g., jdk1.8.0_172.jdk/Contents/Home/lib & jdk1.8.0_172.jdk/Contents/Home/jre/lib/ext/).
    So thanks again!
    So do you think this finds all the services published by Java itself too? Or do I have to find those some other way?
    I think I'd have to feed the resource implementations found this way back into package specific searches using your tool. For example, I did this before:
    Dave Wichers
    @davewichers
    Classes implementing abstract class: javax.xml.parsers.SAXParserFactory
    com.sun.xml.internal.xsom.impl.parser.SAXParserFactoryAdaptor
    org.apache.xerces.jaxp.SAXParserFactoryImpl
    And the search through the META-INF/services/ folders of all the libraries found: org.apache.xerces.jaxp.SAXParserFactoryImpl.
    But it of course didn't find: com.sun.xml.internal.xsom.impl.parser.SAXParserFactoryAdaptor, because that's built into Java itself. So, now I have to feed each abstract class name found that a service was published for back into a specific search for all implementations of that class so it would find them in both Java itself, and all the libraries. Not super hard, but it would be super cool (for me anyway) if there was a magic method that did all this in one shot.
    I can send the code I wrote to do the library search and extract out just the class name and the implementation classes if you are interested. Email might be a better mechanism for such transmission.
    Dave Wichers
    @davewichers
    Now that I think about it more, I believe there has to be a way to find the services implementations published within Java itself. Do you know how? For example, I know that: com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl is an implementation of: javax.xml.transform.TransformerFactory. And a number of XML parsing libraries implement it too.
    Any idea how to simply dump ALL the service implementations published by Java itself so I can compare that list to what my libraries publish/override?
    Luke Hutchison
    @lukehutch

    Great, I'm glad you were able to find all the services you were looking for.

    So do you think this finds all the services published by Java itself too? Or do I have to find those some other way?

    Well ClassGraph goes to extreme lengths to support all classpath and module path specification mechanisms in existence. It has by far the widest support of any library for all these weird and complicated ways of adding to the classpath and module path.

    https://github.com/classgraph/classgraph/wiki/Classpath-Specification-Mechanisms

    However note that since you're looking in system-wide lib and ext directories, if you want to find classfiles in there, you need to call .enableSystemJarsAndModules() and/or .acceptLibOrExtJars() before .scan() to read things from these dirs. This is probably why you didn't find this one, since rt.jar is not scanned by default, unless you call .enableSystemJarsAndModules():

    But it of course didn't find: com.sun.xml.internal.xsom.impl.parser.SAXParserFactoryAdaptor, because that's built into Java itself

    You asked a question earlier that I neglected to answer. If you wanted to first discover all services, then find superclasses or subclasses of those service classes (or similar), you could scan the classpath twice: once to find the files in META-INF/services, then once to find classes that are related to those classes in some way.

    Or if you just need a ClassInfo reference for each of the service classes, so that you can query those classes, you can do it with a single scan by simply adding .enableAllInfo() to read all class metadata for all classes on the classpath or module path -- you can read resources and then ClassInfo references from a single ScanResult. So just enable everything, and don't call .acceptPaths("META-INF/services"), but rather after the scan is complete, call .getResourcesMatchingPattern("META-INF/services/.*"). Then read all the class names from the service files. Then for each one, call scanResult.getClassInfo(className). Then with each class, you can find out super/subclasses, interfaces, etc. -- and you could for example find out the name of the superclass, then find all other names of subclasses of that superclass, if that was useful.

    I believe there has to be a way to find the services implementations published within Java itself

    ServiceLoader has iterator() and stream() methods that let you list all registered service providers for a given service provider interface, however as far as I know, there's no way to find all ServiceLoader instances for all service provider interfaces. I could easily be wrong though, I haven't used this mechanism much.

    I think if you need all of them, ClassGraph is probably your best bet for finding them.

    Dave Wichers
    @davewichers
    OK. I'll see what I can accomplish given your suggestions. Thanks for the detailed response!
    Giuseppe Barbieri
    @elect86

    hey folks,
    I have an interface with an abstract type

    public interface SciJavaPlugin {
        Class<? extends SciJavaPlugin> type();

    And then I have some implementing interfaces, such as

    public interface Gateway extends SciJavaPlugin {
        @Override
        default Class<? extends SciJavaPlugin> type() {
            return AbstractConverter.class;
        }

    Is there a way to retrieve that AbstractConverter.class with ClassGraph?

    Luke Hutchison
    @lukehutch
    Hi @elect86 -- ClassGraph can only read static class metadata, it doesn't read class bytecode at all. The example you gave has the class referred to by bytecode, so ClassGraph cannot read this value. The only way to get the value is to execute the code. ClassGraph can find the type() method, however, and it can find the Gateway class, by looking for all interfaces that extend SciJavaPlugin. It can then load the Gateway class, and you can then just call the type() method yourself to get the class reference.
    One other possibility should be to store the class reference in a static final field, e.g. public static final Class<? extends SciJavaPlugin> type = AbstractConverter.class; -- and then ClassGraph should be able to read the value as a constant initializer value. However I tested this previously, and the Java compiler (at least as of Java 8, when I tested it) does not actually store class references as constant field initializer values. I don't know why, but it probably has something to do with classloading and linking being dynamic processes.
    https://github.com/classgraph/classgraph/blob/latest/src/main/java/io/github/classgraph/Classfile.java#L899
    Therefore, if you tried to do this, you would probably just get null for the field initializer value. And I think the method of using ClassGraph to find and load the class, then manually calling type(), is the only option.
    Giuseppe Barbieri
    @elect86
    @lukehutch thanks for the elaborate reply
    so, if I want to retrieve it with ClassGraph my only way is to go back to Annotations, or?
    Luke Hutchison
    @lukehutch
    Do you understand how to find the method with ClassGraph? ClassGraph can find the method, and you can just call it to get the value. You don't need annotations. You can't get the value without executing the method. The way to find the method is to find interfaces that extend the interface you are interested in, then find the method in that interface.
    Giuseppe Barbieri
    @elect86
    yeah, the fact is that I wanted to avoid loaded the class, I'm trying to stick to a lazy strategy
    But I guess I can figure things out by looking at the superclasses
    Luke Hutchison
    @lukehutch
    To do what you want to do, you'll need to use ObjectWeb ASM, and actually parse the bytecode of the method body.
    Giuseppe Barbieri
    @elect86
    interesting, thanks for the tip Luke, I'll look into it
    skotos999
    @skotos999
    hey there! I've got a question about the acceptPackages / acceptPackagesNonRecursive. When I add a 8-deep packagename it scans everything upto this packagename in a reverse treewalking fashion. The scan result will contain useless packages from my point of view. I don't care about the root ("") or the level1,2...7 deep packages. What am I doing wrong?
    Luke Hutchison
    @lukehutch
    @skotos999 not sure... please provide a compete example.
    skotos999
    @skotos999

    @lukehutch Unfortunately I'm not allowed to share the exact code with you, but it looks something like this. I want to scan only the given package and get the package info to check if it has a specific annotation or not then collect them in two separate collections.

    try (ScanResult scanResult = new ClassGraph().enableAnnotationInfo().acceptPackagesNonRecursive("com.mydomain.lvl3.lvl4.lvl5.lvl6.lvl7.lvl8").scan()) {
        scanResult.getPackageInfo().forEach(pi -> System.out.println(pi.getName()));
    }

    I haven't add the .hasAnnotation() while iterating through the PackageInfoList. I just wanted to check what's the result of the scan. (because it was really slow in production) The result is:
    "", "com", "com.mydomain", "com.mydomain.lvl3", com.mydomain.lvl3.lvl4", com.mydomain.lvl3.lvl4_sibling", etc... upto "com.mydomain.lvl3.lvl4.lvl5.lvl6.lvl7.lvl8".

    Luke Hutchison
    @lukehutch
    @skotos999 if I remember right, getPackageInfo() always returns all parent packages, even if they were not scanned. I should probably change that to represent just packages that were scanned.
    @skotos999 please add .verbose() before .scan() to see exactly what is being scanned. The main reason for long scan times is massive classpaths, where the goal is to scan only one jar on the entire classpath. You can also verify in the logs whether ClassGraph is scanning other packages it shouldn't be.
    Dmitrii Tikhomirov
    @treblereel
    Are there any known issues running ClassGraph from AnnotationProcessor ? looks like it ignores some of my classes
    nafg
    @nafg
    Hi, is it possible to find all callers of a method? (What I really want is a list of all public methods that have no caller)
    (please @-mention me in replies so I notice)
    Derek Perez
    @perezd
    Testing question: Is there a way to produce a ClassInfo from a Class<?> It'd make it easier for testing contexts instead of mocking ClassInfo.
    Luke Hutchison
    @lukehutch
    Sorry everyone, for some reason I wasn't getting notifications from this channel
    @treblereel I haven't seen that before -- can you please file a bug and provide a small testcase? I haven't used AnnotationProcessor, so I don't know how to set that up
    Dmitrii Tikhomirov
    @treblereel
    hmmm, ok, my test cases is simple: i can't get all implementations of a class using ClassGraph. I think, i can do a reproducer (i use it in maven reactor project)
    @lukehutch by the way, thanks for your cool lib!
    Luke Hutchison
    @lukehutch
    @nafg Yes, you can find the whole dependency graph of all classes on the classpath -- see ClassGraph#enableInterClassDependencies(), ClassInfo#getClassDependencies(), ScanResult#getClassDependencyMap(), ScanResult#getReverseClassDependencyMap() -- mentioned briefly here: https://github.com/classgraph/classgraph/wiki/ClassInfo-API
    @treblereel You're welcome! Please do create a reproducer, I'm interested in fixing any and all bugs.
    @perezd The easiest way is to call scanResult.getClassInfo(cls.getName), but I suspect what you're saying is that that doesn't work for your testing context. It should be possible to add a constructor public ClassInfo(Class<?> cls) that populated as many fields of the ClassInfo as possible using reflection. Would you be willing to take a first shot at that?

    General announcement: I just released ClassGraph version 4.8.117. This version uses some nefarious methods to completely circumvent the strong encapsulation that the JDK team has decided to enforce by default in JDK 16+. You can see the details here: https://github.com/classgraph/classgraph/releases/tag/classgraph-4.8.117

    Please report any regressions -- thanks.

    nafg
    @nafg
    @lukehutch I don't see there where to get a method's callers
    Rob Oxspring
    @roxspring
    It seems that enableSystemJarsAndModules() and overrideClassLoaders() are mutually exclusive. Is this by design or just an omission?
    Rob Oxspring
    @roxspring
    I need to scan a specified classpath, including how they depend on system classes, but excluding my own code and dependencies. Looking at ClasspathFinder it seems that scanning modules is disabled if an override is given for classpath or classloader.
    Bojan Tomić
    @kaqqao
    @lukehutch I was looking into various classpath scanning libraries, and it struck me as interesting that not a single project opted to implement the javax.lang.model API. So I'm wondering if you've considered that at any point?
    Bojan Tomić
    @kaqqao
    Btw, just out of curiosity, do you know what Jandex does to discover the paths to scan and if it's significantly different from ClassGraph in that regard? Since it's an ongoing struggle against JDK getting closed up ever more tightly...
    Marc Magon
    @GedMarc
    @kaqqao jandex has major issues with the new class structures, jdk9 and up, only use classgraph in my opinion -
    On our environment we've replaced all the class scanners from wildfly, hibernate, cdi, faces, onto classgraph (making only one scan cycle per application), everything else we did, every scanner, just doesn't work, or works at a performance level that is not acceptable
    Dmitrii Tikhomirov
    @treblereel
    hmm, looks like i missed something and ClassGraph().enableAllInfo().scan() ignores classes from src/test/java with maven. Maybe i need to add something to my pom.xml ?
    Dmitrii Tikhomirov
    @treblereel
    i run classGraph from my apt processor