Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    yxtwang
    @yxtwang
    hello，Anyone present here,When will snappydata support spark2.3? Do you have a plan？thx。
    Ishaq Sahibole
    @ishaq1189_gitlab
    This is Ishaq.. Newly started a POC with SnappyData to replace existing Spark and Cassandra cluster in our Company.. However, when we worked on SnappyData Streaming, we could see that JVM Heap memory is getting increased like anything for a small testcase and also we could see that all stream data captured in Cache which can be visible from SPARK CACHE Tab in pulse UI. Could anyone please explain why stream data is getting cached in SPARK Cache and when it will be cleared from cache. Also, how the memory management should be for the snappydata streaming?
    Jags Ramnarayan
    @jramnara
    Can you share your program? Are u using structured steaming? If so I wouldn’t expect the behavior u see.
    Suds Menon
    @sudsmenon
    @yxtwang Actively working to support Spark 2.3 and beyond in the next release. Engineering effort in full swing. Release announcement as soon as beta is ready
    U.Shaw
    @xyxiaoyou
    @sudsmenon Hi, Suds ! Could you tell us What the approximate time is for the release of the next version?
    girishpatil2193
    @girishpatil2193
    Hi, can any one please help me out, to install snappydata on already installed spark cluster, I have tried this one https://snappydatainc.github.io/snappydata/quickstart/getting_started_with_your_spark_distribution/
    girishpatil2193
    @girishpatil2193
    after this step, I haven't find any file like sbin/snappy-start-all.sh
    or any other snappydata required files
    please send me solution to install snappydata on existing spark cluster?
    U.Shaw
    @xyxiaoyou
    Can SnappyData support more than 1TB of data query? Has anyone tested it?
    U.Shaw
    @xyxiaoyou
    When I use '...shop_id IN (SELECT shop_id FROM tmp_auth)...' to query data, it reports exception:'com.pivotal.gemfirexd.internal.engine.distributed.message.LeadNodeExecutorMsg$SparkExceptionWrapper: java.lang.IndexOutOfBoundsException: 1'. But, when I use ' ...shop_id IN (SELECT shop_id FROM tmp_auth ORDER BY SHOP_ID )... ', it works. Sometimes, if 'order by' is not added and 'limit' is the only one, the result returned by the row table query will be different. @jramnara Can you help me explain this problem? I don't want to lose faith in SnappyData.
    U.Shaw
    @xyxiaoyou
    image.png
    image.png
    When I add ‘order by‘ to the SQL statement and return different results, why?
    U.Shaw
    @xyxiaoyou
    dm_base is column table ,hierarchy_tag_dimension is row table.
    U.Shaw
    @xyxiaoyou
    @sudsmenon
    @swatisawant
    U.Shaw
    @xyxiaoyou
    Solved by set snappydata.sql.disableHashJoin = true
    dawsongzhao
    @dawsongzhao
    hello everyone. who know how to buy Enterprise version of SnappyData and what is it price?
    吴伟杰
    @TeslaCN
    When I using SnappyData's PreparedStatement and set parameter value with non-ASCII (such as Chinese) characters, SnappyData return Nothing.
    select * from table where value = '粤' works normally
    but PreparedStatement " select * from table where value = ? " , set value '粤' into the placeholder it doesn't work.(SnappyData return Nothing)
    Jags Ramnarayan
    @jramnara
    There were several fixes related to PreparedStatement in the upcoming 1.1.1 release. To be released very soon. [SNAP-3082][SNAP-3065][SNAP-2254][SNAP-2644][SNAP-2765][SNAP-3007]
    Chang chen
    @baibaichen
    Hi according cached plan, i noticed that it will reuse rdd, but spark will cache shuffle data internally . how does snappydata handle this ?
    Jags Ramnarayan
    @jramnara

    here is the response from @sumwale -- Shuffle RDDs in the plan are searched in the DAG and explicitly cleared in background after execution. If there is another collect of the same cached plan then any pending clears are first waited on.

    Now that the same problem exists in Spark where it will use the previous cached shuffle result if the same plan is collected multiple times. For example if one does collect() on the same DataFrame multiple times then obsolete results can be returned. So in Spark one should execute the query everytime.

    dawsongzhao
    @dawsongzhao
    hello everyone. who know how to add comment of filed of row table?
    Viacheslav Tradunskyi
    @Tradunsky

    Hi folks,
    Trying to load a table form multiple paths on s3 via SQL: https://jira.snappydata.io/browse/SNAP-3331

    Can somebody suggest simple workaround for now? I can think only of using scala instead of SQL.

    tazone
    @tazone
    Hello everyone, it is any way to create structured streaming pyspark app and submit it to continous execution on snappydata cluster ?
    I want to see in snappy dashboard in section Structured Streaming my structured streaming query