Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Aug 24 12:26

    beniz on master

    API for doupled weight decay : … Merge pull request #627 from fa… (compare)

  • Aug 24 12:26
    beniz closed #627
  • Aug 24 12:25

    beniz on master

    dd api for caffe warmup Merge pull request #623 from fa… (compare)

  • Aug 24 12:25
    beniz closed #623
  • Aug 24 12:24

    beniz on master

    API options for lookahead Merge pull request #621 from fa… (compare)

  • Aug 24 12:24
    beniz closed #621
  • Aug 24 11:17
    beniz labeled #611
  • Aug 24 11:17
    beniz labeled #619
  • Aug 24 11:17
    beniz labeled #619
  • Aug 24 11:16
    beniz labeled #621
  • Aug 24 11:16
    beniz labeled #621
  • Aug 24 11:16
    beniz labeled #623
  • Aug 24 11:16
    beniz labeled #623
  • Aug 24 11:15
    beniz labeled #627
  • Aug 24 11:15
    beniz labeled #627
  • Aug 24 11:15
    beniz labeled #627
  • Aug 24 11:09
    beniz commented #567
  • Aug 23 17:32
    BynaryCobweb commented #611
  • Aug 23 14:37
    fantes synchronize #627
  • Aug 23 14:36
    BynaryCobweb synchronize #611
cchadowitz-pf
@cchadowitz-pf
that's the service_start_list piece
for service creation, it does
calls_output.push_back(service_create(sname,body));
i can probably create a patch/MR for what I imagine it should be doing instead, if you'd like
(i.e. check the result of each service_create and service_predict call and if it's not good, return the failure and stop)
Emmanuel Benazera
@beniz
ok yes, obviously this code is not checking return statuses
this is because the exceptions are caught within each call, e.g. service_create
"simple" way would be to check on the JDoc output object, and check on reported status.
cchadowitz-pf
@cchadowitz-pf
right - but in theory if it's not a dd_created_201 then it should just forward the bad status and stop there, so shouldn't be too difficult i'd imagine
Emmanuel Benazera
@beniz
FYI, JDoc is kinda annoying to work with, but nothing too harsh.
Now, note that many services would most likely fail at first predict call
This is why we included service_predict as well, to load models up.
cchadowitz-pf
@cchadowitz-pf
right - I think that if either the create or the predict call fails, the entire init should fail
Emmanuel Benazera
@beniz
agreed, but now I remember why we didn't do that :)
This is because we would have to unwind whatever went well and undo it :)
cchadowitz-pf
@cchadowitz-pf
hah - I was envisioning that the entire process would simply die if one failed :)
Emmanuel Benazera
@beniz
you can't really take that as a given
so at least the final JSON should return the errors if any
which I don't believe it does
and error check + cleanup script (autostart :) ), should be external to DD I think
cchadowitz-pf
@cchadowitz-pf

you can't really take that as a given

why is that? if the start list file itself is a bad file or something like that, the entire dede process stops. wouldn't this fall under a similar scenario? something about the start list file itself is causing an error when creating services, so stop the entire dede process

Emmanuel Benazera
@beniz
nop
at least we can't do that
but I see the point
what I mean by nop is that in our use cases we'd never kill DD
cchadowitz-pf
@cchadowitz-pf
how would you recover from a failed service creation then, if the service creation occurs on DD startup?
Emmanuel Benazera
@beniz
Current semantics are service autostart, not server autostart.
cchadowitz-pf
@cchadowitz-pf
hmm I see
Emmanuel Benazera
@beniz
To recover from failed init, I'd recommend restarting the process, aka docker or anything else
Now, I do see your point
It could be an option to the autostart call
cchadowitz-pf
@cchadowitz-pf

To recover from failed init, I'd recommend restarting the process, aka docker or anything else

Hah, exactly - we do that currently, but with this option there isn't a good way to detect if there was a failed init :)

It could be an option to the autostart call

I like this thought, though

Emmanuel Benazera
@beniz
but that would put the starting of DD and its ending not at the same level, which I don't like too much, but as long as I'm not forced to use it, I'm fine :)
cchadowitz-pf
@cchadowitz-pf
not sure I understand that last bit?
Emmanuel Benazera
@beniz
meaning, personnally, I like having related operations stay at the same 'level' of an architecture, so that control is all handled at the same place
starting DD / stopping DD for instance, but I'm no expert in these matters, probably just a way for me to structure code and scripts etc...
especially life/death of processes. but yes, the option to autostart should work
cchadowitz-pf
@cchadowitz-pf
heh understood - i guess i'm from the perspective that if something is auto starting, and that process fails in some way, it should stop rather than be in a potential broken state. i see your perspective though. maybe i'll see if i can add an option to autostart and PR
Emmanuel Benazera
@beniz
Sure. It's because you are using it at server startup which is fine. Though the JSONApi class can be used from straight C++ for other things as well, without the HTTP server on top of it, so I guess it makes sense to not exit by default.
cchadowitz-pf
@cchadowitz-pf
Ohh I see. I thought the autostart was only ever just a command line startup parameter. didn't think about using it straight from C++ etc.
Emmanuel Benazera
@beniz
It's true that when starting ./dede --service-autostart, it'd make sense to fail start on any failure
cchadowitz-pf
@cchadowitz-pf
:+1: that's my use case :)
Emmanuel Benazera
@beniz
yeah, so I guess you are right, it could be default when using dede
as the http server I mean
cchadowitz-pf
@cchadowitz-pf
right
cchadowitz-pf
@cchadowitz-pf
yikes, I see what you mean about JDoc/rapidjson
thought I could do some simple JDoc compared to dd_create_201() or something
Emmanuel Benazera
@beniz
sure, it's not that difficult, just annoying
cchadowitz-pf
@cchadowitz-pf
opened a PR, happy to make tweaks if it's not in line with what you're looking for :)
Emmanuel Benazera
@beniz
all coming back from vacation, @cchadowitz-pf we'll have your PR pass the tests, then merge
cchadowitz-pf
@cchadowitz-pf
:+1: no worries, thanks!