spring-projects/spring-security

I'm trying to enable spring security only on the actuator endpoints, and allow all other requests, regardless of auth scheme to pass into the app code (which handles security in it's own regard).

Right now this is what I have:

    @Bean
    fun securityWebFilterChain(http: ServerHttpSecurity): SecurityWebFilterChain {
        // NOTE: This will only add spring security to the actuator endpoints, all other endpoints will be handled
        // by the JWT auth handling.
        // https://stackoverflow.com/questions/38403740/authenticate-only-selected-rest-end-points-spring-boot
        return http.authorizeExchange()
            .pathMatchers("/actuator/**").authenticated()
            .anyExchange().permitAll()
            .and().httpBasic()
            .and().csrf().disable()
            .build()
    }

    @Bean
    fun userDetailsService(): MapReactiveUserDetailsService {
        return MapReactiveUserDetailsService(
            User.withDefaultPasswordEncoder()
                .username(actuatorAdminUsername)
                .password(actuatorAdminPassword)
                .roles("SUPERUSER")
                .build()
        )
    }

This works to ensure that:
1) http basic auth can be used to access the actuator endpoints
2) any bearer request is passed through into the app code (which is what I want)

but doesn't allow an http basic auth request to pass through and be handled by my app code. I've tried a few different incantations to get it working as expected, but mostly get the current state or I get it to pass through http basic requests (at which case my auth on the actuator endpoints doesn't work). Any pointers?