Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Tim van Eijndhoven
    @timve
    As far as I know there is no specific reason, my guess would be that we started the project using the old version and started using the new version at a later point in time without modifiying the older code. The whole use of Rx in the project isn’t very consistend and idiomatic anyway I guess.
    Wolfgang Fahl
    @WolfgangFahl
    @timve thx for the swift reply. It looks like a version mess now:
        <rxjava.version>3.0.0</rxjava.version>
        <!-- asynchttpclient uses rxjava2 -->
        <asynchttpclient.version>2.10.5</asynchttpclient.version>
        <!-- rxjava string uses rxjava 1-->
        <rxjava-string.version>1.1.1</rxjava-string.version>
    Tim van Eijndhoven
    @timve
    personally I would stick with the Rx version supplied by Vert.x, which would be easiest for the code running in Vert.x
    Wolfgang Fahl
    @WolfgangFahl
    I figure the same and try 2.2.18 now. I have to fix the ImageFetcher for the tests to run without errors again.
    I have to replace
    /**
       * start
       * 
       * @param subscriber
       */
      public void start() {
        {
          boolean hasNext = true;
          while (hasNext && !closed) {
            final Image image = this.fetch();
            final Mat frame = image != null ? image.getFrame() : null;
            final Size size = frame != null ? frame.size() : new Size(0, 0);
            hasNext = image != null && size.width > 0 && size.height > 0;
            if (hasNext) {
              if (debug && frameIndex % Math.round(getFps()) == 0) {
                String msg = String.format("->%6d:%4.0fx%4.0f", frameIndex,
                    size.width, size.height);
                LOG.info(msg);
              }
              subject.onNext(image);
            }
          }
          subject.onComplete();
        }
      }
    Wolfgang Fahl
    @WolfgangFahl
    /**
       * convert me to an observable
       * 
       * @return an Image emitting Observable
       */
      public Observable<Image> toObservable() {
        Observable<Image> observable = Observable
            .create(new ObservableOnSubscribe<Image>() {
              @Override
              public void subscribe(ObservableEmitter<Image> observableEmitter)
                  throws Exception {
                boolean hasNext = true;
                while (hasNext && !closed) {
                  final Image image = ImageFetcher.this.fetch();
                  final Mat frame = image != null ? image.getFrame() : null;
                  final Size size = frame != null ? frame.size() : new Size(0, 0);
                  hasNext = image != null && size.width > 0 && size.height > 0;
                  if (hasNext) {
                    if (debug && frameIndex % Math.round(getFps()) == 0) {
                      String msg = String.format("->%6d:%4.0fx%4.0f", frameIndex,
                          size.width, size.height);
                      LOG.info(msg);
                    }
                    observableEmitter.onNext(image);
                  }
                }
                observableEmitter.onComplete();
              }
            });
        return observable;
      }
    Wolfgang Fahl
    @WolfgangFahl
    grafik.png
    Success - the JavaFX app can now work with the simulators images ...
    @timve it would be great if we could do an online session to improve the steering algorithm - with the simulator testing shoud be much easier now ...
    Wolfgang Fahl
    @WolfgangFahl
    There is quite a bit of progress now in using the JavaFX App for debugging the StraightLaneNavigator. Unfortunately there is a nasty bug when communicating with the dash2 simulator:
    In one instance this bug shows after 55 simulator images have been received. One reason for the issue might be that the simulator images have a far too high resolution and might eat up quite a bit of bandwidth and lead to cpu load and latency.
    Wolfgang Fahl
    @WolfgangFahl
    reducing simulator image resolution from retina 2880x1800 to 1054x507 works around this issue
    Wolfgang Fahl
    @WolfgangFahl

    Now the debugging of the navigation begins. With the simulation of a straight road i get courseRelativeToHorizon=-100 all the time from this code:

         * determine the course relative to Horizon
         * @return
         */
        public double determineCourseRelativeToHorizon() {
            Optional<Line> middle = determineLaneMiddle();
    
            if (!middle.isPresent()) {
                return Double.NaN;
            }
    
            Line middleLine = middle.get();
            Point2D middleAtHorizon = middleLine.topMost();
            double middleAtHorizonX = middleAtHorizon.getX();
    
    
            double maxX = 600; // width of image;
    
            double courseAbs = (middleAtHorizonX / maxX);
    
            double courseMaxLeft = 0.7; // 0.7 = auto rijdt max links tov horizon
            double courseMiddle = 0.5; // 0.5 = midden
            double courseMaxRight = 0.2; // 0.2 = auto rijdt max rechts tov horizon
    
    
            double course = 0;
    
            if (courseAbs > courseMiddle) {
                // course left
                double rangeLeft = courseMaxLeft - courseMiddle;
                double courseLeftPercentage = 100 * (courseAbs - courseMiddle) / rangeLeft;
                if (courseLeftPercentage > 100) {
                    courseLeftPercentage = 100;
                }
                course = -courseLeftPercentage;
            }
    
            if (courseAbs < courseMiddle) {
                // course right
                double rangeRight = courseMiddle - courseMaxRight;
                double courseRightPercentage = 100 * (courseMiddle - courseAbs) / rangeRight;
                course = courseRightPercentage;
            }
    
            return course;
    
        }

    I e.g. wonder whether
    double maxX = 600; // width of image; is valid

    Wolfgang Fahl
    @WolfgangFahl
    What is the reasoning behind this code?
        public double determineCurrentAngle() {
            Optional<Line> middle = determineLaneMiddle();
            Optional<Line> left = lane.getLeftBoundary();
            Optional<Line> right = lane.getRightBoundary();
    
            Double angLeft = null;
            Double angMiddle = null;
            Double angRight = null;
    
            if (middle.isPresent()) {
                angMiddle = middle.get().angleDeg() + 90;
            }
            if (left.isPresent() && left.get().length() > 0) {
                angLeft = left.get().angleDeg() + 90 - 40;
                angLeft = angLeft * 1.2;
            }
            if (right.isPresent() && right.get().length() > 0) {
                angRight = right.get().angleDeg() + 90 + 50;
                angRight = angRight * 1.2;
            }
    
    
    
            // debug code
            /*
            int l, m, r;
            l = -1;
            m = -1;
            r = -1;
    
            if (angLeft != null) {
                l = angLeft.intValue();
            }
            if (angMiddle != null) {
                m = angMiddle.intValue();
            }
            if (angRight != null) {
                r = angRight.intValue();
            }
            // System.out.println("l:" + l +", m:"+m + ", r:" + r);
            */
    
            Double result = null;
            if (angMiddle != null) {
                result = angMiddle;
            }
            else if (angRight != null) {
                result = angRight;
            } else if (angLeft != null) {
                result = angLeft;
            } else {
                result = Double.NaN;
            }
    
            // System.out.println("result: " + result);
            return result;
    
        }
    Wolfgang Fahl
    @WolfgangFahl
    @WolfgangFahl
    Another two weeks and still:
    WARN i.v.c.e.i.clustered.ConnectionHolder - Connecting to server localhost:51967 failed
    How can i debug this and find out the reason?
    Why is a connection to localhost tried which does not make any sense since the cluster has been configured otherwise.
    Wolfgang Fahl
    @WolfgangFahl

    The improved ClusterStarter with it's

       * configure the cluster
       * @param clusterHostname
       * @param publicHost
       */
      public void configureCluster(String clusterHostname,String publicHost) {
        if (clusterHostname == null) {
          clusterHostname=getHostname();
        }
        if (publicHost==null) {
          publicHost=getHostname();
        }
        String msg=String.format("starting cluster on %s setting host to %s and clusterPublicHost to %s",getHostname(),clusterHostname,publicHost);
        LOG.info(msg);
        EventBusOptions eventBusOptions = getOptions().getEventBusOptions();
        // https://github.com/eclipse-vertx/vert.x/issues/3229
        // https://stackoverflow.com/a/49028531/1497139
        // https://vertx.io/docs/apidocs/io/vertx/core/eventbus/EventBusOptions.html#setClusterPublicHost-java.lang.String-
        eventBusOptions.setHost(clusterHostname);
        eventBusOptions.setClusterPublicHost(publicHost);
      }

    is now used so that bi-directional communication is possible.

    Tim van Eijndhoven
    @timve
    The double maxX = 600; // width of image; needs to have the value of the amount of horizontal pixels in the image as the other values are calculated in respect to the image width (eg, the desired horizon point X value percentage would be 50%, in an image of 1000 pixels wide this would mean at an absolute value of 500, in an image of 600 pixels wide this would mean an absolute value of 300, etc.)
    (but you probably figured that out already)
    Tim van Eijndhoven
    @timve

    I believe the determineCurrentAngle()code is a remnant from an earlier steering algorithm based on the angle the car has with respect to the road (the premise is that it is desirable that the car should be straight on the road).

    This is calculated by taking the angles of the detected lines and using that to determine the angle the car is under. By preference this is done with respect to the middle line as this takes into account both left and right lines (the middle line is only present when both sides are detected), if one of the side lines is not present the algorithm falls back to determining the angle based on the line that is present in the image.

    I think the angleDeg() calculates the angle with regards to the horizontal direction at the bottom of the image:

    ——————>

    resulting in a vertical line being at -90 degrees (hence the +90)

    Tim van Eijndhoven
    @timve
    The left and right lines are always at an angle (because of the perspective) therefore the when the car is straight these lines would still be at an angle >0. That is what the - 40 and + 50 are for. (the values were probably determined by trial and error, them being different might be the result of the camera not being exactly straight)
    The * 1.2 that is applied when only detecting the left or right line might be a multiplication of the angle to initiate a more severe course correction as the detection of only one side could be the result of the car being very close to one of the road edges, but that is just speculation.
    Tim van Eijndhoven
    @timve
    @WolfgangFahl Too be honest I do not have much input at the moment for improving the steering algorithm, one of the most common comments @bertjan and I received from our talks was to implement some sort of PID loop to tune the steering (to reduce the “drunk driving” effect)
    Wolfgang Fahl
    @WolfgangFahl
    @timve thx for the feedback.
    Yesterday and today i refactored the code so that https://github.com/rc-dukes/dukes/blob/master/rc-detect/src/test/java/org/rcdukes/detect/TestLaneDetection.java succeeds and the standard JsonObject mapping can be used for LaneDetectionResult. The three lines are now fully transmitted to the StraightLaneNavigator. I intend to rewrite the StraighLaneNavigator and make sure it will use the LaneDetectionResult's history for determining where to steer. Also i want to change the command so steer by angle instead of servo positions. I am trying to get rid of dead code while doing so. One candidate is the Point3D support which i think might not be necessary. Also I think it would be simpler to use a open source geometry library like https://github.com/dlegland/javaGeom - but for the time being i just want to make sure that things will run next week at JavaLand (if it will happen see https://www.javaland.eu/de/home/)
    A peculiar behavior is that the LaneDetection gives different results on travis than on my laptop. What could be the reason for that?
    Tim van Eijndhoven
    @timve
    I wrote the geometry stuff myself as an excercise (I had not used proper math since university), switching to an open source library would be the logical thing to do when new geometrical functions are required.
    Tim van Eijndhoven
    @timve
    I cannot see an obvious reason why LaneDetection would give different results on travis than on your laptop… does it behave consistently on your laptop when running via maven and from the IDE? (I have seen cases where tests broke on CI because of parallelism, which mostly became evident when running tests locally using the same maven command as CI)
    Wolfgang Fahl
    @WolfgangFahl
    @timve for the time being i'll not look into the difference issue. Instead rc-dukes/dukes@e4a260f has the servoAngle handling. This is much more compatible to the dash2 simulator so testing with the simulator gets much more feasible.
       * determine the course relative to Horizon
       * 
       * @return
       */
      public Double determineCourseRelativeToHorizon() {
        if (middle == null) {
          return null;
        }
        Point2D middleAtHorizon = middle.topMost();
        double middleAtHorizonX = middleAtHorizon.getX();
        double maxX = viewPort.getWidth();
        // @TODO - distance to horizon is taken as 1 here ...
        double tan = (middleAtHorizonX / maxX);
        return Math.atan(tan);
      }
    is the simplified course calculation. I doubt that it is correct ...
    Tim van Eijndhoven
    @timve
    if distance to horizon is taken as 1 the angle would be way smaller than it actually is? This type of calculation should be quite accurate when considering a birds eye view, it is influenced by the perspective of the camera when considering the forward view (I am not sure how big the influence actually is though)
    Wolfgang Fahl
    @WolfgangFahl
    Unfortunately JavaLand is not going to happen next week. Still I am working on improving the navigation. Today I introduced the time aspect by adding all LaneDetectionResults to a TinkerGraph and later evaluate it to find all entries for a given time Window. These are then statistically analyzed for min,max,average and standard deviation. Fun fact for the http://wiki.bitplan.com/videos/full_run.mp4 is that the car should simply stay straight ... see e.g. result after frame 30:
    30
    Mar 11, 2020 6:05:42 PM org.rcdukes.action.StraightLaneNavigator
    INFORMATION: middle:  -10.5° <-   -0.8° ±    3.9° ->    4.1° /  24 in 1000 msecs
    Mar 11, 2020 6:05:42 PM org.rcdukes.action.StraightLaneNavigator
    INFORMATION:   left:   56.9° <-   59.2° ±    1.6° ->   61.6° /  24 in 1000 msecs
    Mar 11, 2020 6:05:42 PM org.rcdukes.action.StraightLaneNavigator
    INFORMATION:  right:  -62.7° <-  -59.6° ±    1.0° ->  -58.6° /  24 in 1000 msecs
    Mar 11, 2020 6:05:42 PM org.rcdukes.action.StraightLaneNavigator
    INFORMATION: course:   -0.1° <-   -0.0° ±    0.0° ->    0.0° /  24 in 1000 msecs
    Wolfgang Fahl
    @WolfgangFahl
    @bertjan as dicusssed in our chat on the weekend i have added rc-dukes/dukes#59 as new issue
    Wolfgang Fahl
    @WolfgangFahl
    @timve and @bertjan Could you please review the following code whether you think it would work in principle:
     /**
       * process the laneDetectResult
       * 
       * @param ldr
       *          - the lane detection result
       * @return - a message to be sent to the vehicle or null on error
       */
      @Override
      public JsonObject getNavigationInstruction(LaneDetectionResult ldr) {
        // empty message signals no navigation
        JsonObject message = null;
    
        // if in emergency stop mode do not continue
        if (this.emergencyStopActivated)
          return message;
        // set the current time and start time
        setTime(ldr);
        // add the lane detection result to the tinker graph
        addToGraph(ldr);
        // analyze the LaneDetectionResults of the relevant time Windows
        int timeWindow = 1000;
        boolean showDebug = true;
        analyzeAngleRanges(timeWindow, showDebug);
    
        // do we need to stop since there have been no lines for too long?
        if (currentTime - startTime > MAX_DURATION_NO_LINES_DETECTED
            && stopRangeLeft.count + stopRangeRight.count == 0
            || emergencyStopActivated) {
          this.emergencyStopActivated = true;
          return ActionVerticle.emergencyStopCommand();
        }
        // if we have enough detections for statistics we'll use this information
        int MIN_FOUND_PER_TIMEWINDOW = 3;
        // decide which angleRange to use
        AngleRange navigateRange = null;
        if (middleRange.count >= MIN_FOUND_PER_TIMEWINDOW) {
          navigateRange = courseRange;
        } else {
          // are we left or right heavy?
          navigateRange = leftRange.count > rightRange.count ? leftRange
              : rightRange;
        }
        // check that we can send a command
        if (this.cmdOk(navigateRange,MIN_FOUND_PER_TIMEWINDOW)) {
          double angle=navigateRange.steer();
          String msg = ldr.debugInfo()
              + String.format("\nsteer by %s: %s", navigateRange.name,Line.angleString(angle));
          LOG.debug(msg);
          tsLatestCommand = currentTime;
          // @TODO check polarity
          message = steerCommand(-(angle));
        }
        return message;
      }
    The AngleRange class is:
    /**
       * staticical analysis of angles in the given past timeWindow
       * @author wf
       *
       */
      class AngleRange {
        Double min = Double.MAX_VALUE;
        Double max = -Double.MAX_VALUE;
        double sum = 0;
        Double avg = null;
        int count;
        int timeWindow;
        private String name;
        private Double stdDev;
    
        /**
         * create an angle Range for the given vertices
         * 
         * @param name
         * @param currentTime
         * @param timeWindow
         */
        public AngleRange(String name, long currentTime, int timeWindow) {
          this.name = name;
          List<Vertex> angleNodes = g().V().has(name)
              .has("milliTimeStamp", P.gt(currentTime - timeWindow)).toList();
          count = angleNodes.size();
          this.timeWindow = timeWindow;
          if (count == 0) {
            min = null;
            max = null;
            stdDev = null;
          }
    
          for (Vertex angleNode : angleNodes) {
            double angle = (Double) angleNode.property(name).value();
            sum += angle;
            max = Math.max(max, angle);
            min = Math.min(min, angle);
          }
          avg = sum / count;
          double sum2 = 0;
          for (Vertex angleNode : angleNodes) {
            Double angle = (Double) angleNode.property(name).value();
            Double avgDiff = angle - avg;
            sum2 += avgDiff * avgDiff;
          }
          stdDev = Math.sqrt(sum2 / count);
        }
    
        public double steer() {
          double angle=avg;
          return avg;
        }
    
        /**
         * get the debug info
         * 
         * @return
         */
        public String debugInfo() {
          String info = String.format("%6s: %s <- %s ± %s -> %s / %3d in %3d msecs",
              name, Line.angleString(min), Line.angleString(avg),
              Line.angleString(stdDev), Line.angleString(max), count, timeWindow);
          return info;
        }
      }
    So we either steer by courseToHorizon if we have enough middle angles or by left or right angle. I'd love to add the rest of the logic to the steer function e.g. check polarity and adjust steering to the statistical situation, current speed feedback from sensors and the like ...
    Wolfgang Fahl
    @WolfgangFahl
    Milestone 0.0.3 release is in the works the module overview in https://github.com/rc-dukes/dukes has been updated
    Bert Jan Schrijver
    @bertjan

    So we either steer by courseToHorizon if we have enough middle angles or by left or right angle.

    Exactly. The challenge with steering by left or right angle is to not "oversteer" and make the car swerve

    In our case we were probably steering away from the sides too aggresively
    Wolfgang Fahl
    @WolfgangFahl
    @bertjan thx for the feedback. I just added:
    /**
       * receive an car position
       * 
       * @param message
       */
      protected void receiveCarPosition(Message<JsonObject> message) {
        JsonObject carPosJo = message.body();
        // this.eventbusLogger.log(carPosJo.encodePrettily());
        ServoPosition carPos = carPosJo.mapTo(ServoPosition.class);
        if (positionDisplay != null)
          this.positionDisplay.showPosition(carPos);
        if (this.getNavigator()!=null) {
          this.getNavigator().addVertex(carPos);
        }
      }
    the addVertex(carPos) adds the replied current car position to the graph database so that the steering effect can be checked. E.g. when you try to steer 30 degrees you'll know that the car might only have done 14 degrees.
    Wolfgang Fahl
    @WolfgangFahl
    ssh picaro date +%s;date +%s
    Warning: No xauth data; using fake authentication data for X11 forwarding.
    1584107077
    1584107077
    Wolfgang Fahl
    @WolfgangFahl
    https://github.com/naokishibuya/car-finding-lane-lines has a good description on finding lanes
    Wolfgang Fahl
    @WolfgangFahl
    The color filter step seems to be quite useful/important - road color filtered road see http://wiki.bitplan.com/index.php/LaneDetection
      public void testColorFilter() throws Exception {
        ColorFilter cf=new ColorFilter();
        cf.setMinColorRGB( 65,  85,  85);
        cf.setMaxColorRGB(140, 140, 140);
        Mat frame = getTestImage();
        Mat gray=new Mat();
        Imgproc.cvtColor(frame, gray, Imgproc.COLOR_BGR2GRAY);
        assertEquals(2458261,Core.countNonZero(gray));
        Mat colorFiltered = cf.filter(frame);
        assertEquals(colorFiltered.width(), frame.width());
        Mat cfGray=new Mat();
        Imgproc.cvtColor(colorFiltered, cfGray, Imgproc.COLOR_BGR2GRAY);
        assertEquals(173768,Core.countNonZero(cfGray));
        if (show) {
          ImageUtils.show(getTestImage());
          ImageUtils.show(gray);
          ImageUtils.show(colorFiltered); 
        }
      }
    Wolfgang Fahl
    @WolfgangFahl