by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Ananth Durai
    @ananthdurai_twitter

    ```./pinot-0.016-pkg/bin/pinot-admin.sh AddSchema -controllerHost pinot-controller-dev1 -controllerPort 9000 -schemaFile flights.json -exec
    Executing command: AddSchema -controllerHost pinot-controller-dev1 -controllerPort 9000 -schemaFilePath /home/adurai/flights.json -exec
    Caught exception while sending file: flights
    org.apache.commons.httpclient.HttpException: POST Status Code: 500

    at com.linkedin.pinot.common.utils.FileUploadUtils.sendFile(FileUploadUtils.java:148)
    at com.linkedin.pinot.common.utils.FileUploadUtils.sendFile(FileUploadUtils.java:100)
    at com.linkedin.pinot.tools.admin.command.AddSchemaCommand.execute(AddSchemaCommand.java:119)
    at com.linkedin.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:118)
    at com.linkedin.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:130)

    Exception caught:
    org.apache.commons.httpclient.HttpException: POST Status Code: 500

    at com.linkedin.pinot.common.utils.FileUploadUtils.sendFile(FileUploadUtils.java:148)
    at com.linkedin.pinot.common.utils.FileUploadUtils.sendFile(FileUploadUtils.java:100)
    at com.linkedin.pinot.tools.admin.command.AddSchemaCommand.execute(AddSchemaCommand.java:119)
    at com.linkedin.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:118)
    at com.linkedin.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:130)```
    any thoughts on the root cause of it?
    Ananth Durai
    @ananthdurai_twitter
    :point_up: The only exception I see in the controller console is
    java.lang.RuntimeException: Https server CA Certificate file not confugured (ssl.server.ca-cert) at com.linkedin.pinot.common.segment.fetcher.HttpsSegmentFetcher.setupTrustManagers(HttpsSegmentFetcher.java:120) at com.linkedin.pinot.common.segment.fetcher.HttpsSegmentFetcher.init(HttpsSegmentFetcher.java:100) at com.linkedin.pinot.common.segment.fetcher.SegmentFetcherFactory.initSegmentFetcherFactory(SegmentFetcherFactory.java:116) at com.linkedin.pinot.controller.ControllerStarter.start(ControllerStarter.java:119) at com.linkedin.pinot.tools.admin.command.StartControllerCommand.execute(StartControllerCommand.java:156) at com.linkedin.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:118) at com.linkedin.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:130) Starting Pinot Helix resource manager and connecting to Zookeeper
    I guess this is not the root cause of the issue
    Subbu Subramaniam
    @mcvsubbu
    @ananthdurai_twitter this exception should also say that the https protocol will be disabled, that shold be ok.
    If you pull the latest code that includes linkedin/pinot@d7d6d0f then you should be able to configure any protocols you don't use to NoOpFetcher, and you should not see any exceptions
    For the 500 return on schema, there should be a log on the controller, no?
    Ananth Durai
    @ananthdurai_twitter
    nope, it is just throwing the exception on client. I see nothing unusual on the controller log.
    2017/12/15 15:11:37.738 INFO [com.linkedin.pinot.controller.ControllerStarter] [] Controller services available at http://pinot-controller-dev1:9000/ 2017/12/15 15:12:25.801 INFO [com.linkedin.pinot.controller.helix.core.retention.RetentionManager] [] Trying to run retentionManager! 2017/12/15 15:12:25.803 INFO [com.linkedin.pinot.controller.helix.core.retention.RetentionManager] [] Finished update deletion strategies for entire cluster! 2017/12/15 15:12:25.804 INFO [com.linkedin.pinot.controller.helix.core.retention.RetentionManager] [] Finished update segment metadata for entire cluster! 2017/12/15 15:12:25.804 INFO [com.linkedin.pinot.controller.helix.core.retention.RetentionManager] [] Finished segment purge for entire cluster! 2017/12/15 15:12:25.804 WARN [com.linkedin.pinot.controller.helix.core.SegmentDeletionManager] [] Deleted segment directory /tmp/PinotController/Deleted_Segments does not exist or it is not directory. 2017/12/15 15:12:25.804 INFO [com.linkedin.pinot.controller.helix.core.retention.RetentionManager] [] Finished remove aged deleted segments! 2017/12/15 15:13:35.816 INFO [com.linkedin.pinot.controller.validation.ValidationManager] [] Starting validation 2017/12/15 15:13:35.817 INFO [com.linkedin.pinot.controller.validation.ValidationManager] [] Validation completed 2017/12/15 15:13:35.850 INFO [com.linkedin.pinot.controller.helix.SegmentStatusChecker] [] Starting Segment Status check for metrics 2017/12/15 15:13:35.852 INFO [com.linkedin.pinot.controller.helix.SegmentStatusChecker] [] Segment status metrics completed in 1ms
    Jean-François Im
    @jfim
    @ananthdurai_twitter kishore told me you want to run Pinot on AWS?
    Ananth Durai
    @ananthdurai_twitter
    @jfim yes that is correct.
    Jean-François Im
    @jfim
    okay, I'll have a link for you in ~5 mins on how to run Pinot on AWS using kubernetes
    for demo purposes, I'll write the S3 storage part sometime next week
    Ananth Durai
    @ananthdurai_twitter
    @jfim hmm, we are not using kubernetes for now. I'm actually trying to get a cluster with ec2 instances to quickly compare the performance.
    Jean-François Im
    @jfim
    I see
    Ananth Durai
    @ananthdurai_twitter
    I tried the pinot docker instance and it works fine. This work is mostly a precursor to chef-fy pinot installation.
    Jean-François Im
    @jfim
    gotcha, I assume you already had an instance of zk and kafka up and running?
    Ananth Durai
    @ananthdurai_twitter
    yes that is correct. I verified that Helix able to access zookeeper.
    Jean-François Im
    @jfim
    gotcha, then it should be pretty straightforwards
    Ananth Durai
    @ananthdurai_twitter
    yeh, well I'm able to run the controller, broker & server. as per the log it is all running fine.
    when I try to add a schema it causing the issue as mentioned above.
    Jean-François Im
    @jfim
    Can you connect to the controller successfully?
    on port 9000
    it should display a webpage
    Ananth Durai
    @ananthdurai_twitter
    ha, nope. I got Request failed. Grizzly 2.3.23
    Jean-François Im
    @jfim
    hahaha
    :poop:
    Ananth Durai
    @ananthdurai_twitter
    :-)
    Jean-François Im
    @jfim
    any errors on the controller console?
    Ananth Durai
    @ananthdurai_twitter
    but the controller console not showing any exceptions.
    Jean-François Im
    @jfim
    weird
    which version of the docker image are you using?
    Ananth Durai
    @ananthdurai_twitter
    oh, this error is not with the docker.
    Jean-François Im
    @jfim
    so what version of the code do you have deployed?
    Ananth Durai
    @ananthdurai_twitter
    I'm trying to install manually in ec2 instance. I package the master branch today morning.
    Jean-François Im
    @jfim
    oh gotcha
    not sure in what state master is currently
    Ananth Durai
    @ananthdurai_twitter
    that remind me my question, is there a branch or tag release policy enabled?
    Jean-François Im
    @jfim
    we cut a release every week or so from master
    that's why there are tags like mp-1.2.3 on the trunk
    Kishore Gopalakrishna
    @kishoreg
    @jfim this looks more like a config issue than a code issue
    Jean-François Im
    @jfim
    Could be but normally the controller page should always come up
    There's no configuration for it other than the port
    Kishore Gopalakrishna
    @kishoreg
    Ananth have you checked the controller page?
    how did you start the controller and did you specify the port?
    Jean-François Im
    @jfim
    okay, I'm going to head home before it gets to prime time, I'll be back in ~15 mins
    Ananth Durai
    @ananthdurai_twitter
    @kishoreg yes.
    ./pinot-0.016-pkg/bin/pinot-admin.sh StartController -controllerHost pinot-controller-dev1 -controllerPort 9000 -zkAddress "druid-zookeeper-dev2:2181" -dataDir "/tmp/PinotController" -clusterName "perf"
    Kishore Gopalakrishna
    @kishoreg
    pinot-controller-dev1 is a DNS name?
    Ananth Durai
    @ananthdurai_twitter
    I can see 2017/12/15 15:11:37.737 INFO [com.linkedin.pinot.controller.api.ControllerAdminApiApplication] [] Start jersey admin API on port: 9000 2017/12/15 15:11:37.737 INFO [com.linkedin.pinot.controller.ControllerStarter] [] Started Jersey API on port 9000 2017/12/15 15:11:37.738 INFO [com.linkedin.pinot.controller.ControllerStarter] [] Pinot controller ready and listening on port 9000 for API requests 2017/12/15 15:11:37.738 INFO [com.linkedin.pinot.controller.ControllerStarter] [] Controller services available at http://pinot-controller-dev1:9000/ 2
    Kishore Gopalakrishna
    @kishoreg
    can you try with actual IP
    just to rule out binding problem