Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jun 19 2018 00:00
    @ledell banned @renatomarinho
Cheng WeI
@valkyrias_gitlab
before it's giving a struct of [p0,p1], now it's giving a string col as 0/1 directly
@michalkurka is this change intentional? I checked the log and didn't see updates directly related. Thanks,
razou
@razou

Hello,
Does the number of features in interaction (list) parameter for glm algorithm limited to 2 ?
When I try to create interactions between more than 2 features I getting the error bellow.

glm_interactions = ["x1", "x2", "x3"]

glm_params = {"standardize": False,
              "intercept": True,
              "max_iterations": 100,
              "lambda_search": True,
              "nlambdas":100,
              "interactions": glm_interactions,
              "early_stopping": True,
              "seed": 45
             }

glm_feats = ['x1’, 'x2', 'x3',  'x4’, 'x5' ]

glm = H2OGeneralizedLinearEstimator(**glm_params)

glm.train(y=response, 
          x=glm_feats,
          training_frame=trainDF, 
          validation_frame=validationDF
         )

stacktrace:

OSError: Job with key $03017f00000132d4ffffffff$_b0f8cabd0d436ef8ff61e4938913459b failed with an exception: java.lang.ArrayIndexOutOfBoundsException: Index 3 out of bounds for length 3
stacktrace: 
java.lang.ArrayIndexOutOfBoundsException: Index 3 out of bounds for length 3
    at hex.DataInfo.filterExpandedColumns(DataInfo.java:506)
    at hex.glm.ComputationState.applyStrongRules(ComputationState.java:275)
    at hex.glm.ComputationState.setLambda(ComputationState.java:160)
    at hex.glm.GLM$GLMDriver.computeSubmodel(GLM.java:1998)
    at hex.glm.GLM$GLMDriver.computeImpl(GLM.java:2104)
    at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:239)
    at hex.glm.GLM$GLMDriver.compute2(GLM.java:807)
    at water.H2O$H2OCountedCompleter.compute(H2O.java:1455)
    at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
    at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
    at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
    at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
    at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
Michal Kurka
@michalkurka

SW version v3.28.0.1-1 is giving different result when using mojo = H2OMOJOModel.createFromMojo(path).transform(df)

lets ask the sparkling water guys @jakubhava

@razou it isn't limited to 2, you are hitting a bug (which is likely specific to your particular dataset)
razou
@razou
Thanks @michalkurka
Jordan Bentley
@jbentleyEG
are there any known issues with sparkling water where it will hang indefinitely when loading a spark frame?
looking at the Spark console it finishes the writerCTX call, and then just hangs
and I don't see the dataframe in the H2O UI
it doesn't happen every time
I'm trying to replicate it today with a higher level of logging
Jordan Bentley
@jbentleyEG
Screen Shot 2020-01-08 at 4.37.35 PM.png
that is where it is hanging
Jordan Bentley
@jbentleyEG
Michal Kurka
@michalkurka
thanks for the bugreport
Jordan Bentley
@jbentleyEG
I added some comments to the bug report, I was able to get it working
although the method makes 0 sense to me
Michal Kurka
@michalkurka
thanks for the details, I will ask @jakubhava or Marek to look into it
Jordan Bentley
@jbentleyEG
thanks!
I have everything working right now, but the behavior is weird so whatever deadlock there was will probably occur again for someone else
I now have a fully operationalized model building pipeline going through H2O and serving out of mojos, so I'm pretty happy
razou
@razou
@jbentleyEG with only h2o or sparkling water (h2o + spark) in distributed mode ?
benoit
@BenoitLondon
hi anyone can help I have a bad_alloc error when using autoML with XGBoost
 cat /tmp/Rtmp98fg08/h2o_***********_started_from_r.err
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by ml.dmlc.xgboost4j.java.NativeLibLoader (file:/home/**********/R/x86_64-suse-linux-gnu-library/3.5/h2o/java/h2o.jar) to field java.lang.ClassLoader.usr_paths
WARNING: Please consider reporting this to the maintainers of ml.dmlc.xgboost4j.java.NativeLibLoader
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
terminate called after throwing an instance of 'thrust::system::detail::bad_alloc'
  what():  std::bad_alloc: out of memory
if I exclude XGBooost it works fine
benoit
@BenoitLondon
even with a small dataset I get this issue so I believe it s not a RAM issue
razou
@razou

Hi I’m using H2O XGBoost with following settings:

  • Cluster size : 3 nodes and 800 GB of memory
  • Data: 50 M rows and 15 columns
  • XGBoost: max_depth=6, ntrees=200, subsample=0.8, eta=0.05

I found it extremely slow compare to other xgboost’s implementations (e.g dmlc): more than 3 hours
Is it normal ?

Thanks

Michal Kurka
@michalkurka
are you comparing distributed impl to a single node impl?
or is it multinode-multinode comparison?
(for a dataset like yours, a single node should be just fine)
razou
@razou
I tested on single node with 400GB of memory but It was also very slow
Michal Kurka
@michalkurka

ok, our benchmarks don't show any such issue; that makes me think it must be something about your setup

What is your H2O version and what does H2O prints when its starts - it should tell you what XGBoost backend is being used, you want to make sure you are using OpenMP, not the single-threaded fallback version.

razou
@razou
  • I using h2o-3.28.0.1
  • I’m h2o via jupyter notebook on AWS cloud and

The way I’m installing h2o: conda install -c h2oai h2o

Here is my code
The output ou H2O start

Checking whether there is an H2O instance running at http://localhost:54321 ..... not found.
Attempting to start a local H2O server...
  Java Version: openjdk version "11.0.5" 2019-10-15; OpenJDK Runtime Environment (build 11.0.5+10-post-Ubuntu-0ubuntu1.118.04); OpenJDK 64-Bit Server VM (build 11.0.5+10-post-Ubuntu-0ubuntu1.118.04, mixed mode, sharing)
  Starting server from /opt/conda/lib/python3.6/site-packages/h2o/backend/bin/h2o.jar
  Ice root: /tmp/tmptc1_6xe2
  JVM stdout: /tmp/tmptc1_6xe2/h2o_jovyan_started_from_python.out
  JVM stderr: /tmp/tmptc1_6xe2/h2o_jovyan_started_from_python.err
  Server is running at http://127.0.0.1:54321
Connecting to H2O server at http://127.0.0.1:54321 ... successful.
H2O cluster uptime:    01 secs
H2O cluster timezone:    Etc/GMT
H2O data parsing timezone:    UTC
H2O cluster version:    3.28.0.1
H2O cluster version age:    1 month
H2O cluster name:    H2O_from_python_jovyan_fmr3t8
H2O cluster total nodes:    1
H2O cluster free memory:    400 Gb
H2O cluster total cores:    64
H2O cluster allowed cores:    64
H2O cluster status:    accepting new members, healthy
H2O connection url:    http://127.0.0.1:54321
H2O connection proxy:    {'http': None, 'https': None}
H2O internal security:    False
H2O API Extensions:    Amazon S3, XGBoost, Algos, AutoML, Core V3, TargetEncoder, Core V4
Python version:    3.6.8 final
Youshaa Murhij
@YoushaaMurhij
I am using Driverless AI inside Docker on Windows 10 Pro .. but the problem is that I can not access it through localhost:12345 .. anyone faced the same problem?
razou
@razou
@YoushaaMurhij same problem on AWS where I’m running h2o is running inside docker container
Michal Malohlava
@mmalohlava

@YoushaaMurhij @razou did you expose the port 12345 via docker -p option?
@razou Please also make sure that you security group assigned to your instance allows TCP port 12345?

btw: there is a Slack community where you can ask Driverless AI related questions: https://www.h2o.ai/slack-community/

razou
@razou
Thanks @mmalohlava
My security group allows the port 12345 but I’ don’t know if the port was exposed, i’ll check this
razou
@razou

Hi, does anybody faced this version mismatch error before ?

pip uninstall h2o
pip install https://h2o-release.s3.amazonaws.com/h2o/rel-yu/2/Python/h2o-3.28.0.2-py2.py3-none-any.whl
import h2o
h2o.init()

I’m getting this error

H2OConnectionError: Version mismatch. H2O is version 3.28.0.2, but the h2o-python package is version 3.28.0.1. Install the matching h2o-Python version from - http://h2o-release.s3.amazonaws.com/h2o/rel-yu/2/index.html.
Honza Sterba
@honzasterba
@razou that is a strange behaviour, what is the output of h2o.init()?
razou
@razou
the output of h2o.init() is the error above
Honza Sterba
@honzasterba
in the output above I can see " to H2O server at http://127.0.0.1:54321 ... successful" and no H2OConnectionError, what am I missing?
razou
@razou

oh, that’s two different things, when I say above I mean this one:

H2OConnectionError: Version mismatch. H2O is version 3.28.0.2, but the h2o-python package is version 3.28.0.1. Install the matching h2o-Python version from - http://h2o-release.s3.amazonaws.com/h2o/rel-yu/2/index.html.

only my last post (starting from pip uninstall h2o)

angela0xdata
@angela0xdata
Can you launch Flow and make sure that it was shut down properly?
razou
@razou
It’s always an EMR new cluster different from the preious on (not same machines)
Michal Kurka
@michalkurka
@razou the python package is built correctly (I verified that), can you provide the full output that ends with "Version mismatch” error?
Jacob Niebloom
@niebloomj
does anyone know anything about this error? I am running h2o on emr with spark. Thank you!
01-22 20:14:33.229 17479  #59:54321 ERRR: java.lang.ArrayIndexOutOfBoundsException: 13
01-22 20:14:33.229 17479  #59:54321 ERRR:     at water.TCPReceiverThread.basic_packet_handling(TCPReceiverThread.java:307)
01-22 20:14:33.229 17479  #59:54321 ERRR:     at water.TCPReceiverThread$UDP_TCP_ReaderThread.run(TCPReceiverThread.java:249)
01-22 20:14:33.236 17479  #18:54321 WARN: UDP packets from outside the cloud: 1/sec, last one from /:54321 @ Wed Jan 22 20:14:33 UTC 2020
01-22 20:14:33.236 17479  #83:54321 WARN: UDP packets from outside the cloud: 2/sec, last one from /:54321 @ Wed Jan 22 20:14:33 UTC 2020
01-22 20:14:33.240 17479  #69:54321 ERRR: java.lang.ArrayIndexOutOfBoundsException: 13
01-22 20:14:33.240 17479  #69:54321 ERRR:     at water.TCPReceiverThread.basic_packet_handling(TCPReceiverThread.java:307)
01-22 20:14:33.240 17479  #69:54321 ERRR:     at water.TCPReceiverThread$UDP_TCP_ReaderThread.run(TCPReceiverThread.java:249)
01-22 20:14:33.241 17479  #82:54321 ERRR: java.lang.ArrayIndexOutOfBoundsException: 16
01-22 20:14:33.241 17479  #82:54321 ERRR:     at water.TCPReceiverThread.basic_packet_handling(TCPReceiverThread.java:307)
01-22 20:14:33.241 17479  #82:54321 ERRR:     at water.TCPReceiverThread$UDP_TCP_ReaderThread.run(TCPReceiverThread.java:249)
01-22 20:14:33.242 17479  #50:54321 ERRR: java.lang.UnsupportedOperationException
01-22 20:14:33.242 17479  #50:54321 ERRR:     at java.nio.ByteBuffer.array(ByteBuffer.java:994)
01-22 20:14:33.242 17479  #50:54321 ERRR:     at water.H2ONode$UDP_TCP_SendThread.run(H2ONode.java:556)
razou
@razou
@michalkurka @honzasterba here is the complete output https://imgur.com/284Oovc
razou
@razou
@michalkurka @honzasterba I don’t know how and why, but the problem disappears (today, on new emr cluster) and this the following works correctly
pip uninstall h2o
pip install https://h2o-release.s3.amazonaws.com/h2o/rel-yu/2/Python/h2o-3.28.0.2-py2.py3-none-any.whl
if h2o.cluster():
    h2o.cluster().shutdown()

h2o.init()
Matthew J. Oldach
@moldach

Hello, I'm not sure if this is best posted here or as an issue in one of the h20ai github repos (if so please let me know where the best place to post question related to H2oai for R).

I'm trying to get the leaf indices that each sample was assigned to in the forest from h2o.randomForest() (as one can get from sklearn's ExtraTreesClassifier or from R's randomForest and ranger).

library(h2o)
h2o.init(nthreads = 1)

# convert data to h2o objects
train <- as.h2o(d_train)
test <- as.h2o(d_test)

# Convert response to factor (required by randomForest)
train[,ycol] <- as.factor(train[,ycol])
test[,ycol] <- as.factor(test[,ycol])

system.time(
  model <- h2o.randomForest(
    x = xcols,
    y = ycol,
    training_frame = train,
    seed = 123,
    ntrees = 100  ### which parameter do I need to get leaf indices assigned to each sample?
    )
  )

I had made a more detailed post on StackOverflow but haven't gotten any feedback so I wanted to reach out directly to the devs.

Thank you