Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Rushikesh Ghatpande
    @rushigerrard
    Yes, I have 30, 50 gb datasets... I would much prefer scriptable approach than using ECLWatch
    johnholt
    @johnholt
    OK. I’ll find out how to add your system into the landing zone list or to disable the check. Another alternative that would not require a configuration change is to use the SOAP services interface to the ESP which would preserve the scripting but is more complicated.
    johnholt
    @johnholt
    Rushikesh Ghatpande
    @rushigerrard
    Yes. It helped. Thanks.

    Working with flat text file :
    I sprayed a file as a csv with no delimiters and no quotes. The first 2 lines would be posted in the 1st reply.

    I also passed the maxRecordSize as 2147483646 (1 less than INT.MAX) because of the large record size / line.

    The ECL Record structure is :
    RECORD
    STRING field1;
    END;

    I had the following questions :

    1. Def : UNKNOWN char field1
      Why UNKNOWN?
      2.XML :
      <?xml version="1.0"?>

      <Table>
      <Field ecltype="string" label="field1" name="field1" position="0" rawtype="-983036" size="-15" type="string"></Field>
      <filename>file1</filename>
      </Table>

    Why is the rawtype negative?

    My corresponding ECL code is as follows:

    CODE: SELECT ALL
    dataset_name := '~benchmark::file1';
    rs := RECORD
    STRING field1;
    END;

    histogram_dataset2 := DATASET(dataset_name, rs, THOR);

    PATTERN histogramRule1 := PATTERN('(?<=_)[1]');
    RULE userRatingRule1 := histogramRule1;
    matchText1 := {STRING100 out3 := MATCHTEXT(userRatingRule1)};
    userRatingAnswerOne := PARSE(histogram_dataset2, field1, userRatingRule1, matchText1, BEST, MANY, NOCASE);

    ecl run HistogramRating.ecl --target=thor --server=master:8010

    W20180502-145535 failed

    <Result>
    <Exception><Code>4294967295</Code><Source>eclagent</Source><Message>System error: -1: Graph graph1[1], diskread[2]: SLAVE #1 [172.31.15.1:20100]: CFileSerialStream::get read past end of stream, CFileSerialStream::get read past end of stream - handling file: /var/lib/HPCCSystems/hpcc-data/thor/benchmark/file1._1_of_2 - caused by (-1, CFileSerialStream::get read past end of stream)</Message></Exception>
    </Result>

    How to resolve this error?

    johnholt
    @johnholt
    Do you mean no field delimiters or no record delimiters or both?
    If “~benchmark::file1” is the file that you sprayed as CSV, you need read it as CSV in the DATASET(…, CSV(…)) and not DATASET(…, THOR)
    RicardoFernandez-UY
    @RicardoFernandez-UY
    Hi, I'm trying to install HPCCSystems on ubuntu 20.04, but it fails because Dali uses port 7070 by default, and it is already in use by other application. I tried changing the port in the environment.xml and restarting the system, but it insists in connecting to port 7070, and it show the corresponding error in the dali log. Does anyone know how to solve this? Thank you!
    Here is the error: 0000000F USR 2021-11-24 13:07:03.776 3866411 3866411 "DASERVER[0] starting - listening to port 7070"
    00000010 OPR 2021-11-24 13:07:03.778 3866411 3866411 "ERROR: -7: daserver.cpp(817) : Exception : port in use
    RicardoFernandez-UY
    @RicardoFernandez-UY
    Anybody?
    Gordon Smith
    @GordonSmith
    @RicardoFernandez-UY - if you checkout the "Using Configuration Manager" online documentation: https://hpccsystems.com/training/documentation/installation-and-administration it should show you the way....
    RicardoFernandez-UY
    @RicardoFernandez-UY
    @GordonSmith thank you Gordon!! I believe there is no documentation I haven't tried yet. I've done the configuration steps manually and with the GUI; I've checked that the port has indeed changed in the environment.xml file, restarted the server, cleaned the logs (just in case ;-)), uninstalled, re-installed the whole thing many times, and when Dali starts it goes looking for the default 7070 port and it fails with a timeout, because the port is already in use by other app. Just as an experiment I've done the installation following the same steps as per the manual in a vanilla server (without anything else installed) and it works like a charm. Of course in ther latter it uses the 7070 port which in this case is free.
    Gordon Smith
    @GordonSmith
    I suggest you post your query here: https://hpccsystems.com/bb/ things will be quiet until after folks get back from thanksgiving...
    RicardoFernandez-UY
    @RicardoFernandez-UY
    @GordonSmith Thanks again!!! I've tried. It's a little bit embarrassing, but I forgot my password and the reset password system of the forum is not working (at least not for me). I've tried it ten times, and never received the email with instructions. I've checked the spam folder also, just in case, and nothing. I'm working this problem with someone else and I hope I can solve it soon so I can post the issue where you suggested. It seems the universe is temporarily misaligned for my relationship with HPCCSystems :-) . Warm regards!!
    Gordon Smith
    @GordonSmith
    @RicardoFernandez-UY FYI here is the response I got internally:
    "you cannot via configuration at the moment.
    You can if it is started via the command line with --port <port>"
    I would suggest opening an issue here: https://track.hpccsystems.com/ if this is a serious issue for you?
    RicardoFernandez-UY
    @RicardoFernandez-UY
    @GordonSmith Thank you very much!!! It will become an issue whenever I have to deal with installations with multiple applications already installed, as there is always the possibility of that port being already in use. So, I'll follow your advice and open an issue. However, it is good news that there is a manual workaround for the problem. I'll try that too! I guess in this case I will have to start the components one by one in the correct order to end with the whole platform up and running. Am I right?
    Gordon Smith
    @GordonSmith
    Given Dali is the first component that needs to be started, you could try starting it manually and then use the usual start up service for the rest?
    RicardoFernandez-UY
    @RicardoFernandez-UY
    It sounds quite reasonable. I'll try that and let you know the results. I have no access to the servers until tomorrow.
    RicardoFernandez-UY
    @RicardoFernandez-UY

    @GordonSmith
    Well. This one is giving quite a fight ;-).
    I've tried:

    /opt/HPCCSystems/sbin/hpcc-run.sh -c dali@mydali.service --port 7047 start
    hpcc-run: unrecognized option '--port'
    Failure to parse commandline.

    Then:
    systemctl start dali@mydali.service --port 7047
    systemctl: unrecognized option '--port'

    Then:
    ./dafilesrv --port 7047
    And in the log files I got the usual
    0000000F USR 2021-11-30 21:26:48.958 856557 856557 "DASERVER[0] starting - listening to port 7070"
    00000010 OPR 2021-11-30 21:26:48.960 856557 856557 "ERROR: -7: daserver.cpp(817) : Exception : port in use

    And finally:

    ./daserver--port 7047
    ...
    0000000C USR 2021-11-30 21:46:23.082 858207 858207 "DASERVER[0] starting - listening to port 7070"
    0000000D OPR 2021-11-30 21:46:23.083 858207 858207 "ERROR: -7: daserver.cpp(817) : Exception : port in use
    ...

    I'm not giving up, but I might need some context for where to place the --port option.
    Perhaps the whole command :-)
    Thank you!! And thank you for your patience too!!!

    Rodrigo
    @rpastrana
    joining
    Stuart Ort
    @stuartort
    Welcome @rpastrana
    kenrowland
    @kenrowland
    Joined
    Greg Panagiotatos
    @g-pan
    Howdy!
    Michael Gardner
    @Michael-Gardner
    test
    Anthony Fishbeck
    @afishbeck
    Joined
    David de Hilster
    @dehilsterlexis
    Someone needs to update the HPCC icon
    Gavin Halliday
    @ghalliday
    BTW I'm trying to chase down why jira is down again...
    Stuart Ort
    @stuartort
    I can log into Jira, but when trying to accept and issue and add a comment, the comment section seems to be disabled for me. Anyone else that problem? Actually I can't add a comment anywhere in jira