Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    marvin xu
    @marvinxu-free
    @ancasarb combust/mleap#676
    Anca Sarb
    @ancasarb
    Hi @here, I’ve just released the latest version of mleap (0.16.0), both the scala projects and the pypi package. Release notes are under https://github.com/combust/mleap/blob/master/RELEASE_NOTES.md, thank you all for your contributions and support! Will be updating the documentation in the next few days.
    Igor
    @GoshaP
    Hi, @here. Is there any example how to serialize trained tensorflow model to mleap bundle? Documentation proposes to use tensor flow freeze_graph function. But it's unclear how it can be used to generate mleap bundle.
    Nastasia Saby
    @NastasiaSaby
    Does anyone know how to get back a RandomForest model written in Scikit-Learn please? I would like to use Pyspark to do that. But I can't find a good way to do that.
    Daniel Hen
    @Daniel8hen
    Did anyone ever try to save an XGBoost4J model (as part of a Spark pipeline - Bundle.ml) and load it in a docker for REST API? I'm having some difficulties...
    @ancasarb can you kindly assist?
    Thank you!
    Nastasia Saby
    @NastasiaSaby
    Hello. I'm still stuck. Do you know if it is possible to save a model/pipeline with Scikit-Learn in zip? I can't find a way to do that. Thank you.
    Nastasia Saby
    @NastasiaSaby
    I found a solution. My problem was linked to "databricks". If anyone else is interested, I explained my workaround here: combust/mleap#690
    wyan
    @kungfunerd_twitter
    i am trying to use mleap to log a logistic regression (pyspark.ml.classification.LogisticRegression) model by doing mlflow.mleap.log_model(spark_model=model, sample_input=test_data.limit(1), artifact_path=SAGEMAKER_APP_NAME), and then doing a deploy to SageMaker. But when i use boto3 to make the prediction call, the SageMaker endpoint only returns the prediction label 1 or 0, without the probability value. Is there anywhere i can look into to debug this problem?
    2 replies
    marvin xu
    @marvinxu-free
    java.util.NoSuchElementException: key not found: org.apache.spark.ml.PipelineModel, does anyone met this problem?
    use mleap-spark_2.3.0, seriazation pipelne mode in local environment is success, while failed on cluster mode
    and if it seems reference.conf in mleap-xgboost-spark overwrite reference.conf in mleap-spark
    @Daniel8hen have get any resolution?
    Monark Singh
    @monark789_gitlab

    Hi Guys,

    Is there any way, where we can load models if mleap runtime is added as a dependency in java app? Couldn't figure out from looking at the java doc available.

    Could only find API way of loading the models.

    Andrea Guidi
    @guidiandrea

    Hello everybody :)

    I am new to this chat. I was reading the MLeap documentation and I really think it's a great product. My only concern is that (as far as I know) it's not possible to use Spark-NLP annotators or any other python NLP package. Did anybody manage to build a pipeline with a Lemmatizer or any other processing step which is not included in default Spark ML or sklearn modules?

    mtsol
    @mtsol
    Is there any way of serializing List[String] in a separate folder like being done in DecisionTree and GBT like serializers.
    Alex Holmes
    @alexholmes
    hi folks - is there a rough sense of when JDK11/Spark3 support may be added as per combust/mleap#475 ? thanks so much
    Talal
    @talalryz
    @ancasarb hey, hope you're doing well! Could you take a look at https://github.com/combust/mleap/pull/719/files
    TamilN
    @TamilN

    Hi folks, i am trying to run mleap transform using gRPC, but it fails on the server, can some one help ?

    grpcurl -plaintext -proto grpc.proto -proto mleap.proto -proto bundle.proto -d "$jsg" 9.30.108.176:65328 ml.combust.mleap.pb.Mleap/Transform

    getting this error on the server

    Exception in thread "grpc-default-executor-31" java.lang.Error: java.lang.ClassNotFoundException: json.DefaultFrameReader
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1155)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    Caused by: java.lang.ClassNotFoundException: json.DefaultFrameReader
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    at ml.combust.mleap.runtime.serialization.FrameReader$.apply(FrameReader.scala:20)
    at ml.combust.mleap.grpc.server.GrpcServer.transform(GrpcServer.scala:86)
    at ml.combust.mleap.pb.MleapGrpc

    KaTeX parse error: Can't use function '$' in math mode at position 5: anon$̲9.invoke(MleapG…: anon$9.invoke(MleapGrpc.scala:291)
        at ml.combust.mleap.pb.MleapGrpc
    anon$9.invoke(MleapGrpc.scala:289)
    at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171)
    at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:272)
    at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:653)
    at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
    at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    ... 2 more

    TamilN
    @TamilN
    Hi folks, i have opened a git issue too for above error, its an Internal error if try with ml.combust.mleap.json, https://github.com/combust/mleap/issues/730#issuecomment-740596806
    mtsol
    @mtsol
    Hi all, I opened an issue on mleap
    i want to serialize feature importance vector but mleap is not supporting, is there any resolution?
    combust/mleap#729
    shmyer
    @shmyer
    Hi all, I was wondering if it is intentional that the mleap-spring-boot docker image is only available in snapshot versions? https://hub.docker.com/r/combustml/mleap-spring-boot/tags
    mtsol
    @mtsol
    yes
    i think so
    Anca Sarb
    @ancasarb
    Hey @shmyer I don’t think there is, will make sure that we publish a release version going forward as well.
    Hey @mtsol, will reply on the issue you raised shortly
    mtsol
    @mtsol
    thanks @ancasarb
    shmyer
    @shmyer
    Thanks! @ancasarb
    tellarajesh
    @tellarajesh
    hi folks, just wondering is any road map available, like what features are coming , any plans to support spark3.0 . Also in docs it's mentioned custom transformers adding in Python and C support , is it fully in python or actual code in Scala then make bindings for PySpark?
    himanshusolanki
    @himanshusolanki

    Hi,
    can someone please help me with where exactly(in which file of this project) should I be putting this dependency to use xgboost.

    ml.combust.mleap mleap-xgboost-spark_${scala.binary.version} ${ml.combust.mleap.version}
    a configured example would also be helpful.

    I created the jars using master repo and tried using that but faced this error -
    error: object XGBoostRegressor is not a member of package ml.dmlc.xgboost4j.scala.spark
    import ml.dmlc.xgboost4j.scala.spark.XGBoostRegressor

    I had installed these jars -

    bundle-ml
    mleap-base
    mleap-core
    mleap-executor
    mleap-spark-base
    mleap-spark-extension
    mleap-spark
    mleap-xgboost-runtime
    mleap-xgboost-spark

    Alexis BRENON
    @AlexisBRENON

    Hi. I use MLeap in my project as well as GRPC, and I would like to upgrade ScalaPB to latest version (v0.10.11). However, upgrading it make mleap fail to serialize models that were serializable earlier with the following error:

    A needed class was not found. This could be due to an error in your runpath. Missing class: scalapb/Message
    java.lang.NoClassDefFoundError: scalapb/Message

    So I suppose that this is just a version mismatch between my version of scalaPB and the one expected by MLeap (I see that it still use the v0.7.1.
    Do you see any reason against upgrading scalaPB ? If no, would you appreciate a PR for this ?

    Anca Sarb
    @ancasarb
    hi all, just a small note to say the latest documentation is available at https://combust.github.io/mleap-docs/, thanks!
    anu-srivastava
    @anu-srivastava
    Hi All, Just want to bubble up this issues combust/mleap#475 and wanted to understand by when JDK11/Spark3 support will be added?
    Akarsh Gupta
    @akarsh3007
    How can we change the log level on mleap-serving, I am using the docker-image:0.11.0 ?
    antonkw
    @antonkw
    @seme0021 @hollinwilkins
    hi guys!
    Really quick question. I work with the team that use mleap 0.13, they fixed particular issue there. Is it possible to raise PR against 0.13 to see new minor version (0.13.1)?
    As far as I see all versions are chronologically ordered but still decided to double check if old versions have some non-obvious maintenance.
    Thanks!
    Ryan Vogan
    @voganrc

    Hi @ancasarb

    We think we've found an issue with MLeap's DenseTensor indexing code.

    It doesn't seem to follow row-major or column-major order, and has a different behavior from the SparseTensor indexing found in the same file below.

    Could you take a look at our combust/mleap#760 PR when you get a chance?

    Anca Sarb
    @ancasarb
    Hey sure thing, I’ll take a look tomorrow
    3 replies
    indranilr
    @indranilr

    Any

    Hi All, Just want to bubble up this issues combust/mleap#475 and wanted to understand by when JDK11/Spark3 support will be added?

    Hi @ancasarb Looking for support of Spark 3.1.1, XGboost 1.3.x ,Scala 2.12 and JDK 8 as indicated in this issue : combust/mleap#751 , could you please comment on expected availability of the fix ?

    Anca Sarb
    @ancasarb
    Hey, I’m reviewing that #760 PR should be done in the next day or so. And then I will finish off #475
    austinzh
    @austinzh
    Thanks Anca.
    indranilr
    @indranilr
    Anca, would combust/mleap#751 be solved as well with #475 ?
    mcarb123
    @mcarb123

    Hello! Im trying to serve a simple Pyspark LR model, since Mleap runtime is not available (AFAIK) in python im sending Rest calls to the mleap-spring-boot docker server.

    My model takes a single sparse vector as input, and with some trial/error work I managed to send requests in json format successfully.

    Does anyone know if it's possible to send sparse input in protobuf format? My ideal scenario since latency is a great constraint would be to be able to send gRPC/protobuf inference requests.

    My current body (json) that I could successfully send looks like

    {
       "rows":[
          [
             {
                "indices":[
                   [
                      544
                   ],
                   [
                      3818
                   ]
                ],
                "values":[
                   1,
                   1
                ],
                "dimensions":[
                   615504
                ]
             }
          ]
       ],
       "schema":{
          "fields":[
             {
                "type":{
                   "base":"double",
                   "type":"tensor",
                   "dimensions":[
                      615504
                   ]
                },
                "name":"features"
             }
          ]
       }
    }
    Marian Diaconu
    @neboduus

    Hi Everyone! Do you think is possible to transform a Map[String, FeatureClass] to a DefaultLeapFrame using some kind of automatic mechanism?

    Im using Scala
    The FeatureClass would be :
    FeatureClass(value: String, _type: MyType)

    What Id like to do is to have a dynamic Map (therefore contains a dynamic no of features) that can be transformed into a DefaultLeapFrame that is then fed to the model itself.

    Marian Diaconu
    @neboduus
    anyone here?
    Calin Cocan
    @calinrc
    Hi Everyone. I am trying to import MLeap source project in my IDE but I have a newer JDK in place (Open JDK - 15) and SBT fails resolving 'sun.misc.Cleaner' dependency. I have looked up in the source code and this is used in a single method from Platform class (allocateDirectBuffer) and the method is not used in any place. I might be reading the code incorrectly but in case is not used can be removed and allow MLeap compile (at least) with newer java version than 8?
    Thanks
    austinzh
    @austinzh
    Hi @ancasarb Would you please check if combust/mleap#760 is good to merge? Or if you have further comments, Please let me know.
    Ryan Vogan
    @voganrc
    Also combust/mleap#664 and combust/mleap#763 are small when you get a chance!
    Ryan Vogan
    @voganrc
    Hi @ancasarb, could you take a look at our Spark 3 PR when you get a chance?
    combust/mleap#765
    Arun
    @icarusin
    Hello all. May I know when is mleap v0.19.0 release planned?