Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    BridgingBot
    @GitterIRCbot
    [slack] <pure_interpeter> Currently there is no implementation of transition and instead one of gen. However transition still is getting called
    [slack] <pure_interpeter> How is decided whether transition or gen get called?
    BridgingBot
    @GitterIRCbot
    [slack] <rejuvyesh> I don't think transition is getting called. For multi-agent systems (honestly quite generally) it can be hard to define concrete distributions over the entire state space and easier to define a sampler instead.
    [slack] <Jan Mrkos> The way I see it:
    1. Your problem/model needs to implement transition or gen
    2. The solver then uses either transition or gen to find either full or partial policy, or just a best action fot given state.
      What I thought was that if you use solver that requires gen, but your model defines only transition, then there is a default gen implementation that calls rand(transition(…)) But someone else should confirm.
    [slack] <Jan Mrkos> What is calling transition?
    [slack] <pure_interpeter> Simulate, something anonymous, genout, macro expansion, transition(LastActionWrapper) đź’Ą
    BridgingBot
    @GitterIRCbot
    [slack] <Jan Mrkos> So you are calling simulate on your model with what policy?
    [slack] <pure_interpeter> With a FVMCTS(;coordination_strategy=MaxPlusColneWithModifiedCoordinateActionCall())
    [slack] <Jan Mrkos> Also, what are you actually implementing? An mdp model or a solver?
    [slack] <pure_interpeter> I want implement a solver but the solver needs access to last performed action. So I need a MDP wrapper type that stores that information.
    [slack] <Jan Mrkos> I am not familiar with FVMCTS solver, but it seems strange to me that a MCTS solver should call transition internally.
    BridgingBot
    @GitterIRCbot
    [slack] <Jan Mrkos> What do you mean last performed action? The one that got you to the current state?
    [slack] <Jan Mrkos> The solver needs to generate the next action.
    [slack] <pure_interpeter> From looking into the source of FVMCTS it doesn't call but transition but is uses gen.
    [slack] <Jan Mrkos> Ok, so how do you model that in your MDP? Maybe you should include the last action as a state variable if your next transition depends on it. Otherwise you are breaking the markovian property, no?
    [slack] <Jan Mrkos> Anyway, so what is the solver you are implementing? Its not FVMCTS?
    [slack] <Jan Mrkos> And what is the problem instance you use?
    [slack] <pure_interpeter> The algorithm is called MaxPlusWithCost. I am using the FirstOrderUAV example.
    BridgingBot
    @GitterIRCbot
    [slack] <Jan Mrkos> Ok, and what is implemented in the UAV example? Transition or gen?
    [slack] <pure_interpeter> gen
    [slack] <Jan Mrkos> Ok, then I dont know where the transition comes from :-)
    BridgingBot
    @GitterIRCbot
    [slack] <pure_interpeter> Do you know how to have Julia give longer stack traces?
    [slack] <pure_interpeter> How I model that in my wrapper? when an action is taken I write the action (the action_indexes) into an Array. I can query that state using a private function. All other called get passed through.
    Since there is no method to read state or write problem state I assume the equality relation or hash is used to see if two branches lead to the same state which is not equal between left-up and up-left as the last action store in the wrapper is different
    <system> file IMG_20220201_175725.jpg too big to download (1031897 > allowed size: 1000000)
    [slack] <pure_interpeter> The stack trace is kinda cut off
    [slack] <Jan Mrkos> Idont know about the stacktraces… For me they are usually long enough
    [slack] <pure_interpeter> Thanks.
    BridgingBot
    @GitterIRCbot
    [slack] <Jan Mrkos> I need to put down my phone for now, but I will be on a pc later so I willhave a look. Do you have your code somewhere public? I could have a quick look.
    [slack] <pure_interpeter> Not right now by I put on GitHub later, I'll let you know when it online
    BridgingBot
    @GitterIRCbot
    <system> file IMG_20220201_180858.jpg too big to download (1065114 > allowed size: 1000000)
    [slack] <pure_interpeter> Cleaned up the stack trace by removing the task macros
    BridgingBot
    @GitterIRCbot
    [slack] <rejuvyesh> As the error states, you don't have a proper transition function defined for your LastActionWrapper MDP.
    BridgingBot
    @GitterIRCbot
    [slack] <pure_interpeter> The transition seems to come from node_expr(::Val{:sp}, depargs) in line 106 of gen_impl.jl in POMDPs.jl
    BridgingBot
    @GitterIRCbot
    [slack] <rejuvyesh> Yeah, as I said in a different thread as well as @Jan Mrkos explains, you need either a gen function or transition function defined for your wrapper (it can't be both). Since the underlying MMDP is defined by a gen function I think you need to define an appropriate gen function for your wrapper mmdp.
    BridgingBot
    @GitterIRCbot
    [slack] <pure_interpeter> Ok ... Did function gen and not function POMDPs.gen that might have been the cause
    BridgingBot
    @GitterIRCbot
    [slack] <pure_interpeter> My code in my GitHub freemin7 in the MaxPlusWirhCost branch

    [slack] <pure_interpeter> Usage:
    `using FactoredValueMCTS
    using POMDPs
    using MultiUAVDelivery

    mdp = LastActionWrapper(FirstOrderMultiUAVDelivery())
    solver = FVMCTSSolver(;coordination_strategy=MaxPlusWithCost(agent_action_cost=zeros(11), joint_action_cost=zeros(11,11)))
    planner = solve(solver, mdp)
    s = rand(initialstate(mdp))
    a = action(planner, s)`

    [slack] <pure_interpeter> If you want to have a look and give me some feedback that would be appreciated but it now passes the test suite.
    BridgingBot
    @GitterIRCbot
    [slack] <pure_interpeter> And also thanks a lot
    BridgingBot
    @GitterIRCbot
    [slack] <rejuvyesh> I don't see any LastActionWrapper but should be fine?
    BridgingBot
    @GitterIRCbot
    [slack] <pure_interpeter> I forgot to add it
    BridgingBot
    @GitterIRCbot
    BridgingBot
    @GitterIRCbot
    [slack] <Jan Mrkos> Ok, so I guess you resolved your issue? I had a quick look at the repo but I don't know enough about the domain or the solver to glean anything useful. I can only give generic thoughts that may not apply in your case:
    • I am unsure about the purpose of the wrapper. Conceptually, it seems to me that the last action array should be part of the state, since the next state depends on it. But maybe this is a convention in multi-agent mdps?
    • If I was working on a problem like this, I would try to solve the "simplest possible but still having required properties" problem by hand and make it a test case. Or if that is not possible, at least couple of simpler subproblem (e.g. multi-agent mdp with just one agent, two agents with just one action, ...). I think you need test cases that not only check that the return types are correct, but also that the results are correct. This may be somewhat difficult, but building correctly working algorithm without that may be even harder (at least it would be for me).
    BridgingBot
    @GitterIRCbot

    [slack] <pure_interpeter> Yes it works.

    Conceptually yes it should be. However I actually never need to concatenate the arrays as there is no get/set.
    So I can keep them separated.

    I've considered creating a test to demonstrate that it works.

    BridgingBot
    @GitterIRCbot
    [slack] <Jan Mrkos> I dont understand what you mean by concatenation and get/set. What I meant was a state that looks something like:
    struct State last_action::Array old_state_field_1::Whatever1 old_state_field_2::Whatever2 ... end
    [slack] <Jan Mrkos> Anyway, good on you that you got it working, good luck with your project!
    BridgingBot
    @GitterIRCbot
    [slack] <pure_interpeter> I accomplish just this by storing the subproblem which stores it's own state and doesn't leak it.
    BridgingBot
    @GitterIRCbot

    [slack] <sunbergzach> Hey @Jan Mrkos, @pure_interpeter, and @rejuvyesh, sorry for not monitoring this channel; I was quite busy with the start of the semester. Regarding the original question,

    What is the motivation behind gen and what does it do?
    that is somewhat confusing, so I have added an FAQ: https://juliapomdp.github.io/POMDPs.jl/latest/faq/#What-is-the-difference-between-transition,-gen,-and-@gen? and also updated the main docs about defining problems

    Zachary Sunberg
    @zsunberg
    Hello to anyone participating in the hackathon remotely! I will try to keep an eye on this chat - you can also join the jitsi room via the instructions on the website!
    Julian Galvez Serna
    @Piaktipik
    Hi, here on jitsi