Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Luca Giovagnoli
    @lucagiovagnoli
    @ancasarb thanks so much for sharing your valued experience. I’m not familiar with beans but I’m going to read up about it now :)
    Transformer and RowTransformer is what we’re using, so that sounds great!
    Daniel Hen
    @Daniel8hen
    Hi all, I wanted to ask a junior question :)
    I have a Spark model (XGBoost4J), already serialized in the famous MLeap bundle json. Now I'd like to deploy it to some service on docker / Kubernetes and start querying it. My question is where do I put the parameters that shall be relevant to each request? if I have let's say 1000 features, and only 500 of them are relevant, how should I tackle this use case? Where should I start? the documentation is not that clear about this use case. Thank you!
    prafulrana21
    @prafulrana21
    hi @hollinwilkins , how can i get the list of all the deployed models.
    mtsol
    @mtsol
    @ancasarb How can I replace value of a column in leap frame, by using .withColumn? Or any helpful advice.
    mtsol
    @mtsol
    how can i execute transformation of custom transformer when i have same input and output columns?
    mtsol
    @mtsol
    @ancasarb how can I check if a column exists in a leapframe?
    Gustavo Salazar Torres
    @tavoaqp
    hey guys, I'm working on a Golang library for MLeap, to begin with I'm trying to bring Word2Vec models. So far my problem has been to understand how the JSON model is parsed, is there any documentation about this?
    marvin xu
    @marvinxu-free
    "Failed to find a default value for splits", any one meet this problem while save model with mleap?
    Anca Sarb
    @ancasarb
    @marvinxu-free replied on the github issue
    Anca Sarb
    @ancasarb
    @prafulrana21 at the moment it seems we don’t have an endpoint for that if you’re using the spring boot service (https://github.com/combust/mleap/tree/master/mleap-spring-boot). Let me know if you’re interested in adding one!
    Anca Sarb
    @ancasarb
    @mtsol I’ve replied to the questions here combust/mleap#660, hope it helps!
    marvin xu
    @marvinxu-free
    @ancasarb i have reopen issue, please read it on the github issue.
    marvin xu
    @marvinxu-free
    @ancasarb combust/mleap#676
    Anca Sarb
    @ancasarb
    Hi @here, I’ve just released the latest version of mleap (0.16.0), both the scala projects and the pypi package. Release notes are under https://github.com/combust/mleap/blob/master/RELEASE_NOTES.md, thank you all for your contributions and support! Will be updating the documentation in the next few days.
    Igor
    @GoshaP
    Hi, @here. Is there any example how to serialize trained tensorflow model to mleap bundle? Documentation proposes to use tensor flow freeze_graph function. But it's unclear how it can be used to generate mleap bundle.
    Nastasia Saby
    @NastasiaSaby
    Does anyone know how to get back a RandomForest model written in Scikit-Learn please? I would like to use Pyspark to do that. But I can't find a good way to do that.
    Daniel Hen
    @Daniel8hen
    Did anyone ever try to save an XGBoost4J model (as part of a Spark pipeline - Bundle.ml) and load it in a docker for REST API? I'm having some difficulties...
    @ancasarb can you kindly assist?
    Thank you!
    Nastasia Saby
    @NastasiaSaby
    Hello. I'm still stuck. Do you know if it is possible to save a model/pipeline with Scikit-Learn in zip? I can't find a way to do that. Thank you.
    Nastasia Saby
    @NastasiaSaby
    I found a solution. My problem was linked to "databricks". If anyone else is interested, I explained my workaround here: combust/mleap#690
    wyan
    @kungfunerd_twitter
    i am trying to use mleap to log a logistic regression (pyspark.ml.classification.LogisticRegression) model by doing mlflow.mleap.log_model(spark_model=model, sample_input=test_data.limit(1), artifact_path=SAGEMAKER_APP_NAME), and then doing a deploy to SageMaker. But when i use boto3 to make the prediction call, the SageMaker endpoint only returns the prediction label 1 or 0, without the probability value. Is there anywhere i can look into to debug this problem?
    2 replies
    marvin xu
    @marvinxu-free
    java.util.NoSuchElementException: key not found: org.apache.spark.ml.PipelineModel, does anyone met this problem?
    use mleap-spark_2.3.0, seriazation pipelne mode in local environment is success, while failed on cluster mode
    and if it seems reference.conf in mleap-xgboost-spark overwrite reference.conf in mleap-spark
    @Daniel8hen have get any resolution?
    Monark Singh
    @monark789_gitlab

    Hi Guys,

    Is there any way, where we can load models if mleap runtime is added as a dependency in java app? Couldn't figure out from looking at the java doc available.

    Could only find API way of loading the models.

    Andrea Guidi
    @guidiandrea

    Hello everybody :)

    I am new to this chat. I was reading the MLeap documentation and I really think it's a great product. My only concern is that (as far as I know) it's not possible to use Spark-NLP annotators or any other python NLP package. Did anybody manage to build a pipeline with a Lemmatizer or any other processing step which is not included in default Spark ML or sklearn modules?

    mtsol
    @mtsol
    Is there any way of serializing List[String] in a separate folder like being done in DecisionTree and GBT like serializers.
    Alex Holmes
    @alexholmes
    hi folks - is there a rough sense of when JDK11/Spark3 support may be added as per combust/mleap#475 ? thanks so much
    Talal
    @talalryz
    @ancasarb hey, hope you're doing well! Could you take a look at https://github.com/combust/mleap/pull/719/files
    TamilN
    @TamilN

    Hi folks, i am trying to run mleap transform using gRPC, but it fails on the server, can some one help ?

    grpcurl -plaintext -proto grpc.proto -proto mleap.proto -proto bundle.proto -d "$jsg" 9.30.108.176:65328 ml.combust.mleap.pb.Mleap/Transform

    getting this error on the server

    Exception in thread "grpc-default-executor-31" java.lang.Error: java.lang.ClassNotFoundException: json.DefaultFrameReader
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1155)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    Caused by: java.lang.ClassNotFoundException: json.DefaultFrameReader
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    at ml.combust.mleap.runtime.serialization.FrameReader$.apply(FrameReader.scala:20)
    at ml.combust.mleap.grpc.server.GrpcServer.transform(GrpcServer.scala:86)
    at ml.combust.mleap.pb.MleapGrpc

    KaTeX parse error: Can't use function '$' in math mode at position 5: anon$̲9.invoke(MleapG…: anon$9.invoke(MleapGrpc.scala:291)
        at ml.combust.mleap.pb.MleapGrpc
    anon$9.invoke(MleapGrpc.scala:289)
    at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171)
    at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:272)
    at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:653)
    at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
    at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    ... 2 more

    TamilN
    @TamilN
    Hi folks, i have opened a git issue too for above error, its an Internal error if try with ml.combust.mleap.json, https://github.com/combust/mleap/issues/730#issuecomment-740596806
    mtsol
    @mtsol
    Hi all, I opened an issue on mleap
    i want to serialize feature importance vector but mleap is not supporting, is there any resolution?
    combust/mleap#729
    shmyer
    @shmyer
    Hi all, I was wondering if it is intentional that the mleap-spring-boot docker image is only available in snapshot versions? https://hub.docker.com/r/combustml/mleap-spring-boot/tags
    mtsol
    @mtsol
    yes
    i think so
    Anca Sarb
    @ancasarb
    Hey @shmyer I don’t think there is, will make sure that we publish a release version going forward as well.
    Hey @mtsol, will reply on the issue you raised shortly
    mtsol
    @mtsol
    thanks @ancasarb
    shmyer
    @shmyer
    Thanks! @ancasarb
    tellarajesh
    @tellarajesh
    hi folks, just wondering is any road map available, like what features are coming , any plans to support spark3.0 . Also in docs it's mentioned custom transformers adding in Python and C support , is it fully in python or actual code in Scala then make bindings for PySpark?
    himanshusolanki
    @himanshusolanki

    Hi,
    can someone please help me with where exactly(in which file of this project) should I be putting this dependency to use xgboost.

    ml.combust.mleap mleap-xgboost-spark_${scala.binary.version} ${ml.combust.mleap.version}
    a configured example would also be helpful.

    I created the jars using master repo and tried using that but faced this error -
    error: object XGBoostRegressor is not a member of package ml.dmlc.xgboost4j.scala.spark
    import ml.dmlc.xgboost4j.scala.spark.XGBoostRegressor

    I had installed these jars -

    bundle-ml
    mleap-base
    mleap-core
    mleap-executor
    mleap-spark-base
    mleap-spark-extension
    mleap-spark
    mleap-xgboost-runtime
    mleap-xgboost-spark

    Alexis BRENON
    @AlexisBRENON

    Hi. I use MLeap in my project as well as GRPC, and I would like to upgrade ScalaPB to latest version (v0.10.11). However, upgrading it make mleap fail to serialize models that were serializable earlier with the following error:

    A needed class was not found. This could be due to an error in your runpath. Missing class: scalapb/Message
    java.lang.NoClassDefFoundError: scalapb/Message

    So I suppose that this is just a version mismatch between my version of scalaPB and the one expected by MLeap (I see that it still use the v0.7.1.
    Do you see any reason against upgrading scalaPB ? If no, would you appreciate a PR for this ?

    Anca Sarb
    @ancasarb
    hi all, just a small note to say the latest documentation is available at https://combust.github.io/mleap-docs/, thanks!
    anu-srivastava
    @anu-srivastava
    Hi All, Just want to bubble up this issues combust/mleap#475 and wanted to understand by when JDK11/Spark3 support will be added?
    Akarsh Gupta
    @akarsh3007
    How can we change the log level on mleap-serving, I am using the docker-image:0.11.0 ?
    antonkw
    @antonkw
    @seme0021 @hollinwilkins
    hi guys!
    Really quick question. I work with the team that use mleap 0.13, they fixed particular issue there. Is it possible to raise PR against 0.13 to see new minor version (0.13.1)?
    As far as I see all versions are chronologically ordered but still decided to double check if old versions have some non-obvious maintenance.
    Thanks!
    Ryan Vogan
    @voganrc

    Hi @ancasarb

    We think we've found an issue with MLeap's DenseTensor indexing code.

    It doesn't seem to follow row-major or column-major order, and has a different behavior from the SparseTensor indexing found in the same file below.

    Could you take a look at our combust/mleap#760 PR when you get a chance?

    Anca Sarb
    @ancasarb
    Hey sure thing, I’ll take a look tomorrow
    1 reply