oat++ framework/Lobby

oat++

@oatpp_io_twitter

And finally, is there some recommended way to perform a partial deserialisation? Let's say that I have a json object with one value determining the nature of the rest of its contents, so first I need to extract that one value only, based on that value select the correct DTO, and only after that deserialise the whole object with the correct DTO.

You can use Polymorphic DTO fields for that - https://github.com/oatpp/oatpp/blob/master/changelog/1.3.0.md#polymorphic-dto_field

class MyDto : public oatpp::DTO {

  DTO_INIT(MyDto, DTO)

  DTO_FIELD(String, dataType); //<-- value determining the nature of the rest of its contents
  DTO_FIELD(Any, data); //<-- polymorphic data

  DTO_FIELD_TYPE_SELECTOR(data) {

    if(dataType == "animal") return Object<AnimalDto>::Class::getType();
    if(dataType == "car") return Object<CarDto>::Class::getType();
    if(dataType == "int") return Int32::Class::getType();

    return Void::Class::getType();
  }

};

@mmn:matrix.org

1 reply

@junyama

    auto connectionPool = oatpp::network::ClientConnectionPool::createShared(
      connectionProvider, // connection provider
      10, //max connections
      std::chrono::seconds(5) // max lifetime of idle connection
    );

use ClientConnectionPool::createShared instead

junyama

@junyama

No compile error. Thank you.

mmn

@mmn:matrix.org

[m]

So based on the value of dataType, I would need to select a whole DTO rather than a type for one field.

The obvious dirty solution is to deserialise the received json object twice, in the first round only for dataType and in the second round for all fields with the correct DTO, but of course I'd prefer something that would at least parse the json object only once (and map twice).

junyama

@junyama

This message was deleted

1 reply

axelspoerl

@axelspoerl

Hi, I am creating async listeners with the following code:

OATPP_COMPONENT(std::shared_ptr<std::list<std::shared_ptr<oatpp::network::ServerConnectionProvider>>>, connectionProviders);

std::list<std::thread> threads;

for(auto& provider : *connectionProviders) {
threads.push_back(std::thread([provider]{
OATPP_COMPONENT(std::shared_ptr<oatpp::network::ConnectionHandler>, handler);
oatpp::network::Server server(provider, handler);
server.run();
}));
}

std::thread statThread([]{
printStats();
});

My question is: How do I properly shut the listeners down?

Zach

@acidtonic

If you look at the docs they cover the various start/stop configurations depending on if you need to stop later or not.

oat++

@oatpp_io_twitter

@mmn:matrix.org
So based on the value of dataType, I would need to select a whole DTO rather than a type for one field.
The obvious dirty solution is to deserialise the received json object twice, in the first round only for dataType and in the second round for all fields with the correct DTO, but of course I'd prefer something that would at least parse the json object only once (and map twice).

Yes, in this case, you just have to parse and map it twice.

@axelspoerl
See this example project - https://github.com/oatpp/example-server-stop - it's not yet updated to 1.3.0 but API is almost the same

junyama

@junyama

I have set RetryPolicy as follows:

std::unordered_set<v_int32> m_httpCodes{500};
auto retryPolicy = std::make_shared<oatpp::web::client::SimpleRetryPolicy>(10, std::chrono::seconds(1), m_httpCodes);

Looking at the log of my client code, I saw some strange HTTP response status code, such as 3 and 4. My ENDPOINT is based on CRUD example:

ENDPOINT("POST", "images", createImage, BODY_DTO(Object<ImageDto>, imageDto)) {
      auto userDto = UserDto::createShared();
      serializeImageDto(imageDto, userDto);
      return createDtoResponse(Status::CODE_200, m_imageServices.createUser(userDto));
}

I do not change UserService.cpp. What was wrong? Thank you for your help in advance.

shijiantouzouyiqie

@shijiantouzouyiqie

@oatpp_io_twitter
Hello, question: when the server takes the initiative to send data, F5 refreshes the browser page. The server suddenly reports an error. What's the better way?

Thank you. Here is an example:

class WSInstanceListener : public oatpp::websocket::ConnectionHandler::SocketInstanceListener {
public:
    static bool m_bState;
    ...

};

bool WSInstanceListener::m_bState = true;
void WSInstanceListener::onAfterCreate(const oatpp::websocket::WebSocket& socket, const std::shared_ptr<const ParameterMap>& params) {
  SOCKETS ++;
  OATPP_LOGD(TAG, "New Incoming Connection. Connection count=%d", SOCKETS.load());

  /* In this particular case we create one WSListener per each connection */
  /* Which may be redundant in many cases */
  socket.setListener(std::make_shared<WSListener>());

  m_bState = true;

  std::thread t([&socket]{
    while (m_bState) {
        socket.sendOneFrameText("Hello");
    }
  });

  t.detach();
}

void WSInstanceListener::onBeforeDestroy(const oatpp::websocket::WebSocket& socket) {

  SOCKETS --;
  OATPP_LOGD(TAG, "Connection closed. Connection count=%d", SOCKETS.load());

  m_bState = false;
}

 I |2021-12-09 18:45:59 1639043159419263| MyApp:Server running on port 8000
 D |2021-12-09 18:46:06 1639043166551007| Server_WSListener:onClose code=1001
 D |2021-12-09 18:46:06 1639043166551045| Server_WSInstanceListener:Connection closed. Connection count=0
terminate called after throwing an instance of 'std::runtime_error'
  what():  [oatpp::web::protocol::websocket::WebSocket::sendOneFrameText()]: Unknown error while writing to socket.

oat++

@oatpp_io_twitter

Hello @junyama ,

Please give me more information:

exact errors that you see
client code - how you make the request

17 replies

Hello, question: when the server takes the initiative to send data, F5 refreshes the browser page. The server suddenly reports an error. What's the better way?

Hey @shijiantouzouyiqie , if you refresh the page in web-browser then the client is disconnected from the server, and the server reports that it can't use that already invalid connection.

1 reply

shijiantouzouyiqie

@shijiantouzouyiqie

@oatpp_io_twitter
Hello,after refreshing the browser, the program crashes. How to optimize the error point whenwsinstancelistener::onaftercreate uses std::threadto actively send data?
Or doesstd::threadnot cause the program to crash!

Mo Aboulmagd

@Maboulmagd

Hi I was wondering how to deserialize the following using oatpp::DTO:

"changes": [
    [
      "buy",
      "9122.04",
      "0.00121425"
    ],
    ...,
    [
      "sell",
      "9122.07",
      "0.98942292"
    ]
    ...
  ],

I had hoped I'd be able to do something like this:
Vector<Object<Change>> as opposed to Vector<Vector<String>>>. Ideas?

Also wondering how to do something similar for this:

 "trades": [
      {
          "type": "trade",
          "symbol": "BTCUSD",
          "eventid": 169841458,
          "timestamp": 1560976400428,
          "price": "9122.04",
          "quantity": "0.0073173",
          "side": "sell"
      },
      ...
  ],

Thanks.

Mo Aboulmagd

@Maboulmagd

On another note, what is the advantage of using oatpp DTO's over nlohmann::json?

Zach

@acidtonic

lol I used to work at Bloomberg, so many memories seeing stock data again :)

there exists a special database tailor made for this stuff and they open sourced it... Check out comdb2 which a clusterable sqlite lightweight db optimized for stock data

oat++

@oatpp_io_twitter

Hey @shijiantouzouyiqie ,

Hello,after refreshing the browser, the program crashes. How to optimize the error point whenwsinstancelistener::onaftercreate uses std::threadto actively send data?
Or doesstd::threadnot cause the program to crash!

You can see the error, right? Try to use try - catch

terminate called after throwing an instance of 'std::runtime_error'
  what():  [oatpp::web::protocol::websocket::WebSocket::sendOneFrameText()]: Unknown error while writing to socket.

Also, I wouldn't recommend using my example "as is". It's better to at least pool your threads.

Hey @Maboulmagd ,

Hi I was wondering how to deserialize the following using oatpp::DTO:

"changes": [
    [
      "buy",
      "9122.04",
      "0.00121425"
    ],
    ...,
    [
      "sell",
      "9122.07",
      "0.98942292"
    ]
    ...
  ],

Vector<Vector<String>>>

 "trades": [
      {
          "type": "trade",
          "symbol": "BTCUSD",
          "eventid": 169841458,
          "timestamp": 1560976400428,
          "price": "9122.04",
          "quantity": "0.0073173",
          "side": "sell"
      },
      ...
  ],

Vector<Object<MyObj>>

oat++

@oatpp_io_twitter

On another note, what is the advantage of using oatpp DTO's over nlohmann::json?

One should use whatever serves the need better.

Mo Aboulmagd

@Maboulmagd

@oatpp_io_twitter Makes sense, and thank you, it seems as though I was just not seeing that the data I was getting was mostly strings, as opposed to ints/floats, so my custom DTO's were incorrect.
That said, I guess its worth benchmarking nlohmann::json vs mapper_->readFromString<T>, to see which approach is more performant?
@acidtonic I'm currently working for a trading firm, but this is just a side project for me to see how far I can go, and to learn more...That said, Bloomberg is actually a place I wouldn't mind working at, since its C++, though I do hear that its very team-dependent as to whether you use modern C++, or pre C++11. How was it working there, if you don't mind me asking? Also thanks for the database tip.

2 replies

Zach

@acidtonic

@Maboulmagd if you want to pm me I'll share some opinions.

1 reply

mmn

@mmn:matrix.org

[m]

For prices and quantities... the problem is that they are decimal values that don't necessarily directly map into floats/doubles.

There is an arbitrary-precision type NUMERIC in PostgreSQL, which is a good fit for this type of data. It is apparently not directly supported in oatpp-postgresql, but you can always pass strings to PostgreSQL and it will convert them to the correct types.

mmn

@mmn:matrix.org

[m]

(Well, NUMERIC is AFAIK in standard SQL, the problem is that standard SQL assumes that you specify a fixed decimal scale (PostgreSQL does not)... which might be a problem if you want to have price and quantity data for pairs like Shiba Inu/BTC on one hand, and BTC/Shiba Inu on the other.)

Nikola Radovanovic

@nikoladsp

hi all, short architectural question: say I want to create model as a shared library and use it in multiple micro-services (one service is one process), is it then possible to use single swagger page for all micro-services?

oat++

@oatpp_io_twitter

@nikoladsp ,

You can do that.

You can have a proxy in front of all your services which proxies requests + has the swagger-ui for all upstream services.
Or, you can serve swagger-ui in a separate service. Just make sure that you have the correct server address in swagger-ui and that all your services are accessible at that host-name

Nikola Radovanovic

@nikoladsp

@oatpp_io_twitter , many thanks. I will probably go with second approach

shijiantouzouyiqie

@shijiantouzouyiqie

The browser establishes a link with the server websocket and actively sends data. The browser refreshes the page and closes the page for the first time. The server runs normally; The browser establishes a link with the server websocket and actively sends data. When the browser closes the page for the second time, it will send 1001 status code to the server, and the program crashes;
@oatpp_io_twitter Hello, the following is the current code. Is there a better optimization method?

void* StartThread(void* arg);
class WSInstanceListener : public oatpp::websocket::ConnectionHandler::SocketInstanceListener {
public:

.....
public:

  pthread_t m_pid;
  bool m_bQuitThread;
  oatpp::websocket::WebSocket* m_Socket;

};

WSInstanceListener::WSInstanceListener() {
    m_pid = -1;
    m_bQuitThread = true;
    m_Socket = NULL;
}

void WSInstanceListener::onAfterCreate(
        const oatpp::websocket::WebSocket &socket,
        const std::shared_ptr<const ParameterMap> &params) {
    SOCKETS++;
    OATPP_LOGD(TAG, "New Incoming Connection. Connection count=%d",
            SOCKETS.load());

    socket.setListener(std::make_shared<WSListener>());

    m_Socket = (oatpp::websocket::WebSocket*) &socket;

    pthread_create(&m_pid, NULL, StartThread, (void*) this);
    pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
    pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);
    pthread_detach(m_pid);

}

void WSInstanceListener::onBeforeDestroy(
        const oatpp::websocket::WebSocket &socket) {
    SOCKETS--;
    OATPP_LOGD(TAG, "Connection closed. Connection count=%d", SOCKETS.load());

    m_bQuitThread = false;
    pthread_cancel(m_pid);
    pthread_join(m_pid,NULL);
    this->m_Socket->stopListening();

}

void* StartThread(void *arg) {
    WSInstanceListener *lis = (WSInstanceListener*) arg;

    lis->m_bQuitThread = true;

    while (lis->m_bQuitThread) {
        pthread_testcancel();
        lis->m_Socket->sendOneFrameText("hello");
    }

    return (void*) 0;
}

 I |2021-12-15 19:46:53 1639568813189617| MyApp:Server running on port 8000
 D |2021-12-15 19:47:00 1639568820630942| Server_WSInstanceListener:New Incoming Connection. Connection count=1
 D |2021-12-15 19:47:06 1639568826556318| Server_WSListener:onClose code=1001
 D |2021-12-15 19:47:06 1639568826556353| Server_WSInstanceListener:Connection closed. Connection count=0
 D |2021-12-15 19:47:16 1639568836406451| Server_WSInstanceListener:New Incoming Connection. Connection count=1
 D |2021-12-15 19:47:21 1639568841210745| Server_WSListener:onClose code=1001
 terminate called after throwing an instance of 'std::runtime_error'
 D |2021-12-15 19:47:21 1639568841210782| Server_WSInstanceListener:Connection closed. Connection count=0
 terminate called recursively

AlexandreHURDYK

@AlexandreHURDYK

Hello.

I've got an annoying issue I'm trying to fix on my oat++ client application : if my server shuts down before it gets the time to send a response to my client after a client request, it causes my client to crash. This is probably a common problem. Any way to fix this client side ?

I can provide an example API call from my client if necessary.

There's probably something I forgot to do.

AlexandreHURDYK

@AlexandreHURDYK

Most basic example : if I call an API function while the server is not up, the client crashes.

As simple as that.

I can try to make a minimal working example.

Is there a way to make my request time out or something ?

Ok nevermind I fixed the problem and I'm stupid : the simple solution was just to surround the API calls with a try/catch.

Nikola Radovanovic

@nikoladsp

hi all, advice needed.
for simplicity, say that there is a DTO User class with id (primary key, auto inc, etc), salutation (mr,mrs, default null) and full name (string).

for all crud methods apart create, id is required field. how is it possible to reuse one DTO in such way that swagger know that id is not present at all while creating user object (excluded from schema) and in all other cases is required? or do I need DTO per crud operation. Additionally, during update, id is the only required field, all others are optional, while reading for example I expect id and fullname to be present and salutation is optional.
is it possible to use some pattern (decorator maybe?) to decouple DTO classes from "business" logic classes. I am interested for this especially in cases of relationships (composed objects, where one holds many others)
thanks in advance

mmn

@mmn:matrix.org

[m]

Is there some easy way to determine the reason why oatpp::websocket::WebSocket::listen() returned? It catches the relevant exceptions and does not return any value, which makes it a bit hard.

Okay, maybe I should abandon listen() and "reimplement" it using iterateFrame().

oat++

@oatpp_io_twitter

Hey @nikoladsp ,

In an ideal world you would have to decouple the DB model and REST DTOs. The way you present data to service users and the way you store data may differ (and they do differ in most cases). Also, it's not recommended to expose DB ids - since they do not describe the actual resource in most cases.

Treat CRUD service as an adaptor of REST interface to DB model.

So the answer is - create two sets of DTOs - one for REST, second for DB.
Read from DB - do whatever transformations/compositions you need - join data in memory if needed - form response DTO and return it on the endpoint.

About the 'ID' problem - follow this issue - oatpp/oatpp#485
We may introduce hidden (from swagger docs) fields in future

oat++

@oatpp_io_twitter

Hey @mmn:matrix.org ,

if the WebSocket connection was closed gracefully - you'll receive the onClose method call in WS listener first, then the listen method will return.
If there was any kind of network issue - listen method will just return - no onClose event triggered.
Here is the implementation of listen - https://github.com/oatpp/oatpp-websocket/blob/master/src/oatpp-websocket/WebSocket.cpp#L279

1 reply

Please note that closed connection - is a normal situation for WebSocket - your application has to have the re-connecting functionality.

Also, make sure to respond with pong messages for all pings

Where communities thrive

People

Repo info

Activity