AnyCable/Lobby

Anil Kumar Maurya

@anilmaurya

Hi @palkan
Thank you for Anycable

I am struggling in running cable in production mode. Its working fine in development mode but just hangs when I start with RAILS_ENV=production

I upgraded to latest version of anycable-rails but problem is not resolved.

Vladimir Dementyev

@palkan

Hey! Why do you think it hangs? It’s likely operating normally but not printings logs to the console (which is enabled only in development)

CameloHQ

@CameloHq_twitter

Anyone know how to do interactive debug if anycable is being run through docker-compose and DIP. dip up anycable? I put byebug in the codebase and it does not stop for prompting as similar in Rails server (that running via docker as well).

To clarify, it's Anycable RPC.

CameloHQ

@CameloHq_twitter

For anyone who has the same issue, it's docker-related thing. Try "docker attach [container-id]", and things will work.

Vladimir Dementyev

@palkan

You can add a custom Dip command to enable debugging capabilities: https://github.com/anycable/anycable_rails_demo/blob/c0b8ad885ddbe67209d0bb78cd661ac53ce81cbb/dip.yml#L44-L47

And then you run AnyCable as dip anycable.

Hemant Bhanoo

@hbhanoo

Hi guys. I've been running anycable for a small realtime application. I notice that some of my users who join on their phones (ie. possibly high latency, low bandwidth) 'leave' and 'rejoin' my channels. I'm assuming there's some latency or missed-packet threshhold that they're using that makes them drop and then reconnect. Does anyone know if/how it's possible to tune this?

Vladimir Dementyev

@palkan

Hey! Looks like a reconnection feature of the Action Cable JS client: it assumes that a ping from server is received every 3s; if not it reconnects.
There is no way to change this interval in JS client.
You can try to tune the ping interval at the server side (make it smaller).

also, make sure you're using the latest version of AnyCable-go (at least 1.0.5)

2 replies

Subramanian-ERS

@Subramanian-ERS

Hello Team,
I would like to connect anycable in my Rails application to my Redis cluster hosted on AWS. When I wanted anycable to connect to a single Redis instance, I used the below configuration:
development:
redis_url: <Redis instance URL>

Would it be enough if I just replace the instance URL with the cluster URL? Will anycable have the intelligence to gather the multiple nodes in the cluster and connect to them all?

Vladimir Dementyev

@palkan

Hi!
Redis Cluster is currently not supported. That doesn't make a lot of sense in using a cluster for AnyCable, since we only use pub/sub (currently). So, I'd recommend launching a standalone Redis instance for that.

9 replies

Vladimir Dementyev

@palkan

https://twitter.com/any_cable/status/1405165584564822019

Fodil Youssouf Islam

@fodil-youssouf-islam

Holla everyone,
Hope you all doing well, I just wanna say thank you for your work and contribution to the open-source and ruby community, may God bless you all.
also, I wanna share my experience deploying and using anycable, I had an issue deploying it at heroku (great doc by the way) saying "AnyCable server builder must be configured. Make sure you've required a gem (e.g. grpc) or configured AnyCable.server_builder yourself" and to resolve it I had to add grpc gem to my gemfile which is not needed in other production deployments and locally .

Vladimir Dementyev

@palkan

Thank you!

Yeah, since v1.1 grpc has been declared as an optional dependency; but Ruby doesn’t have such mechanism 🙂

We should think about how to make it easier to figure this out 🤔

rafali

@rafaqat

#12 295.7 compiling ./libsass/src/util_string.cpp

#12 295.7 compiling ./libsass/src/values.cpp

#12 295.7 linking shared-object sassc/libsass.so

#12 295.7 /usr/bin/ld: ast.o: no group info for section

#12 295.7 '.text._ZNK4Sass10VectorizedINS_10SharedImplINS_9StatementEEEE8elementsEv'

#12 295.7 /usr/bin/ld: ast.o: no group info for section

#12 295.7 '.text._ZNK4Sass10VectorizedINS_10SharedImplINS_9StatementEEEE8elementsEv'

#12 295.7 /usr/bin/ld: ast.o: no group info for section

#12 295.7 '.text._ZNK4Sass10VectorizedINS_10SharedImplINS_9StatementEEEE8elementsEv'

#12 295.7 /usr/bin/ld: ast.o: no group info for section

#12 295.7 '.text._ZNK4Sass10VectorizedINS_10SharedImplINS_9StatementEEEE8elementsEv'

#12 295.7 /usr/bin/ld: ast.o: file not recognized: bad value

#12 295.7 collect2: error: ld returned 1 exit status

#12 295.7 make: * [Makefile:262: libsass.so] Error 1

#12 295.7

#12 295.7 make failed, exit code 2

#12 295.7

#12 295.7 Gem files will remain installed in /usr/local/bundle/gems/sassc-2.4.0 for

#12 295.7 inspection.

#12 295.7 Results logged to

#12 295.7 /usr/local/bundle/extensions/x86_64-linux/3.0.0/sassc-2.4.0/gem_make.out

#12 295.7

#12 295.7 An error occurred while installing sassc (2.4.0), and Bundler cannot continue.

#12 295.7 Make sure that gem install sassc -v '2.4.0' --source 'https://rubygems.org/'

#12 295.7 succeeds before bundling.

#12 295.7

#12 295.7 In Gemfile:

#12 295.7 sass-rails was resolved to 6.0.0, which depends on

#12 295.7 sassc-rails was resolved to 2.1.2, which depends on

#12 295.7 sassc

executor failed running [/bin/sh -c bundle install]: exit code: 5
ERROR: Service 'anycable-rpc' failed to build : Build failed

has anyone seen this error before

Vladimir Dementyev

@palkan

There is a problem with libsass compilation; likely, some system deps missing

rafali

@rafaqat

thank you

Guilherme Lages Santos

@guizaols

Hello everyone. I'm facing some difficulties to config the service over here.
I'm running an app using Kubernetes. I've set the RPC deployment, the service and the Go server (all working well when I test using the rails console).
The problem is when I call the url /cable in my frontend application and the response is: No route matches [GET] "/cable".
I'm using the nonAcme config in the helm chart to set the path and host (same as the main rails app).

I'm probably messing around with some of these address.

Any clue?

5 replies

Neven DREAN

@ndrean

Hi! My first experience with AnyCable stopped when running "anycable" . I have an error "could not add port 127.0.0.1:50051 to server, not sure why", "E0811 01:07:21.443241000 4748246464 server_chttp2.cc:49]. Any experience on this?

James O'Meara

@jamesjomeara_twitter

Hi there. I am looking for some advice with regards to deploying AnyCable in a k8s cluster in AWS. Specifically, I am interested in how many anycable-go instances we should be running. From experience, has any one seen an optimal ratio of concurrent users per anycable-go instance? Similar question with the number of RPCs (at least for a starting point); however, I realize this will be different depending on the Rails app / functions it is being asked to perform. Any assistance would be greatly appreciated. Thanks!

Vladimir Dementyev

@palkan

Hi! For a moderate load (up to 5k), a single anycable-go instance would be enough; unless you restart them frequently and want to implement smth like rolling update, of course; in general, I would suggest choosing the pod size (e.g., resource limits) depending the overall cluster utilization; probably, deploying micro-pods wouldn’t be efficient, since you waste the idle RAM for each one.

Regarding RPCs, there are two things to consider: the idle app size (again, RAM) and the throughput.

For the second one, watch for rpc_pending metrics.

1 reply

CameloHQ

@CameloHq_twitter

Hi, just curious while reading Monitor class in the anycable-client package, I did not see anywhere to call dispose to unbind event listeners from watch. Is it a intention?

1 reply

CameloHQ

@CameloHq_twitter

FYI, it's here https://github.com/anycable/anycable-client/blob/master/packages/core/monitor/index.js#L121

Maria Kim

@mkimClio

Hello, I am proposing to use AnyCable but we have some questions regarding the gRPC as we have a high volume of users. What is the connection like from the websocket server to the gRPC? For instance, if we have one websocket connection, does that equal a persistent and open connection to the gRPC? Or is the connection to the gRPC short-lived and on demand?? And what if you have many websocket connections? Does each websocket connection map to a connection to the gRPC? Or is there some sort of pooling? Also, are the outbound connections from the gRPC to the database pooled?

Vladimir Dementyev

@palkan

Hey, Maria!
There is a single gRPC connection; thanks to HTTP2 streams, we can perform concurrent operations efficiently within the same net connection.
At the server side, we have a pool of threads to handle requests. As with other Rails runners, each thread uses its own DB connection (that’s how ActiveRecord works).

2 replies

Maria Kim

@mkimClio

Hi Vladimir! I have a followup question - the go WSS is threaded, but is there still one overall HTTP2 connection to the gRPC server? Or does each thread have one http2 connection to the gRPC server?

Vladimir Dementyev

@palkan

Hey! There is still a single TCP connection under the hood; see also this discussion grpc/grpc-go#682

We've been using a connection pool prior to 1.0, and it turned out to be less performant than using a single connection (yeah, surprisingly).

1 reply

Markos Fragkakis

@markos.fragkakis_gitlab

Hello everyone. We will be probably using AnyCable for our Rails application. I am a bit confused on where the RPC server (the application ran with bundle exec anycable) is invoked. Is it when a client sends a message through a socket (WS server invokes via gRPC the RPC server, which in turn invokes the application)? Or the other way around, when the application broadcasts a message through a socket?

9 replies

Vladimir Dementyev

@palkan

anycable-go v1.1.3 has been released with a performance fix: https://github.com/anycable/anycable-go/releases/tag/v1.1.3

John Bachir

@jjb

Hello! I'm on a Rails 5.2 project and am planning to start using Anycable. I'm also planning on upgrading to Rails 6. Is there any reason to do one before the other?

5 replies

Markos Fragkakis

@markos.fragkakis_gitlab

My understanding of a client subscribing to multiple channels is that we end up with multiple WS connections, not a single one. I didn't see any concept of "multiplexing" multiple channels over single WS connection, could you please verify if my understanding correct?

1 reply

John Bachir

@jjb

I want to use anycable with graphql, which i need Pro for. I'm eager to start testing this. I can't do so until I have access to the pro gem, right? I submitted the form for this but haven't been contacted I believe.

1 reply

John Bachir

@jjb

running bundle exec rails g anycable:setup on an M1 mac

/Users/john/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/bootsnap-1.7.5/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require': dlopen(/Users/john/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/google-protobuf-3.18.0-universal-darwin/lib/google/2.7/protobuf_c.bundle, 9): no suitable image found.  Did find: (LoadError)

2 replies

nweber-bisom

@nweber-bisom

Hi, I have a question about the anycable-rails 1.1.3, when I start the anycable-rails server I receive an issue : "invalid option: --rpc-host=0.0.0.0:50051
". In 1.1.2 it works fine. It is an issue or I need update my config ?
My full start command :
"bundle exec anycable --rpc-host=0.0.0.0:50051 --http-health-port=54321 --http-health-path='/check'"

Vladimir Dementyev

@palkan

Hey!
Could you please check whether you’ve upgraded some other deps as well? Particularly, anyway_config?

I’m asking since there were no functional differences between anycable-rails 1.1.2 and 1.1.3.

2 replies

Bartosz Gaj

@bgaj

Hi, i have problem with large incoming messages. I try to recive large image converted to base64. I change ANYCABLE_MAX_MESSAGE_SIZE but i end up with this error:
error: rpc error: code = desc = Received message larger than max (6994086 vs. 4194304)
Is an option to adjust this grpc config (i am using docker anycable/anycable-go image)

6 replies

Prathap B

@PrathapChronus

Hi Palkan,
We use anycable for more than a year now. Websocket initiation happens only within logged-in pages. So ideally failed_auths_total metric should be 0. But recently we see huge numbers (3000-4000) everyday. (Used to be less than 50 in the inital days).

On investigating, we saw the following pattern of requests:

Sequence 1: Subscription happens and 1 message is sent to a channel every hour (Expected Behaviour)

D 2021-09-20T14:55:14.891Z context=node sid=P-75MnY-zka6bx0TH9Sfj websocket session established
D 2021-09-20T14:55:14.939Z context=node sid=P-75MnY-zka6bx0TH9Sfj Incoming message: &{subscribe {"channel":"AppearanceChannel",..} }
D 2021-09-20T14:55:14.939Z context=node sid=P-75MnY-zka6bx0TH9Sfj Incoming message: &{subscribe {"channel":"NotificationChannel",..} }
D 2021-09-20T15:55:15.469Z context=node sid=P-75MnY-zka6bx0TH9Sfj Incoming message: &{message {"channel":"AppearanceChannel"}... }
D 2021-09-20T16:55:15.469Z context=node sid=P-75MnY-zka6bx0TH9Sfj Incoming message: &{message {"channel":"AppearanceChannel"}... }
.
.
D 2021-09-21T00:56:03.469Z context=node sid=P-75MnY-zka6bx0TH9Sfj Incoming message: &{message {"channel":"AppearanceChannel"}... }

Websocket getting closed with 1005 status (Probably the client system has logged-off/hibernated)

D 2021-09-21T01:23:07.495Z context=node sid=P-75MnY-zka6bx0TH9Sfj Websocket closed: websocket: close 1005 (no status)
D 2021-09-21T01:23:07.495Z context=node sid=P-75MnY-zka6bx0TH9Sfj websocket session completed
D 2021-09-21T01:23:07.495Z context=node sid=P-75MnY-zka6bx0TH9Sfj Disconnect {"wm":"abcde"} http://abc.example.com/cable &map[REMOTE_ADDR:127.0.0.1 ..] [{"channel":"AppearanceChannel"} {"channel":"NotificationChannel"}]
D 2021-09-21T01:23:07.495Z channel={"channel":"NotificationChannel",..} context=hub sid=P-75MnY-zka6bx0TH9Sfj Unsubscribed
D 2021-09-21T01:23:07.495Z context=hub sid=P-75MnY-zka6bx0TH9Sfj Unregistered
D 2021-09-21T01:23:07.504Z context=rpc Disconnect response: status:SUCCESS

Fine till now.
Sequence 2: Things get weirder from now on. We get a connect-disconnect request every minute

D 2021-09-21T01:24:02.936Z context=rpc Authenticate response: status:SUCCESS identifiers:"{\"wm\":\"abcde\"}" transmissions:"{\"type\":\"welcome\"}" env:<cstate:... >
D 2021-09-21T01:24:02.936Z context=node sid=QJ2JeGgAsYzhPv6nQOBJ6 websocket session established
D 2021-09-21T01:24:02.936Z context=hub sid=QJ2JeGgAsYzhPv6nQOBJ6 Registered with identifiers: {"wm":"abcde"}
D 2021-09-21T01:24:02.967Z context=node sid=QJ2JeGgAsYzhPv6nQOBJ6 Websocket closed: websocket: close 1001 (going away)
D 2021-09-21T01:24:02.967Z context=node sid=QJ2JeGgAsYzhPv6nQOBJ6 websocket session completed
D 2021-09-21T01:24:02.967Z context=node sid=QJ2JeGgAsYzhPv6nQOBJ6 Disconnect {"wm":"abcde"} http://abc.example.com/cable &map[REMOTE_ADDR:127.0.0.1 ..] []
D 2021-09-21T01:24:02.967Z context=hub sid=QJ2JeGgAsYzhPv6nQOBJ6 Unregistered
D 2021-09-21T01:24:02.968Z context=rpc Disconnect response: status:SUCCESS

The above connect-disconnect loop occured every minute until 2021-09-28T06:25:08 and then the following

Sequence 3: Authentication failures every minute till today

D 2021-09-28T06:26:08.942Z context=rpc Authenticate response: status:FAILURE transmissions:"{\"type\":\"disconnect\",\"reason\":\"unauthorized\",\"reconnect\":false}" 
E 2021-09-28T06:26:08.942Z context=ws Websocket session initialization failed: Application error:

D 2021-09-28T06:27:08.942Z context=rpc Authenticate response: status:FAILURE transmissions:"{\"type\":\"disconnect\",\"reason\":\"unauthorized\",\"reconnect\":false}" 
E 2021-09-28T06:27:08.942Z context=ws Websocket session initialization failed: Application error:
..
..
..
today

We identified that the above requests are from the same client from NGINX access logs. We also found similar pattern occurred for 2 other IPs as well. We could not reproduce the behaviour. Can you please help us with some insights on when such loops occur?

3 replies

Jeff Keen

@jkeen

I've been trying to resolve the error "undefined method call for ApplicationCable::Connection:Class:", and have gone through every google result and github issue relating to it and still can't find a solution. It works fine on dev (running with overmind) only happens on staging (haven't deployed to prod yet) which is on a VPS configured through hatchbox. The ENV variables seem to be set correctly, but even though ACTION_CABLE_ADAPTER=any_cablesomehow ActionCable.server.config.cable is equal to {"adapter"=>"redis", "url"=>"redis://10.3.96.3:6379/0"} and not {"adapter"=>"any_cable"} like it's supposed to be.

13 replies

If I explicitly hack the config to be any_cable in environments/staging.rb like this

ActionCable.server.config.cable = { "adapter": "any_cable", "url": ENV.fetch("ANYCABLE_REDIS_URL") }.with_indifferent_access

AnyCable::Rails.enabled? returns true like it should, but the that "undefined method call for ApplicationCable::Connection:Class" error persists

Hemant Bhanoo

@hbhanoo

Dumb q. I see a log message that looks like:
[Action Cable] Broadcasting HallChannel-main msg: still (Duration: 1.8395630177110434)
in my logs. If that's in seconds, it seems like a pretty slow broadcast. I would have assumed that rails would just write to redis and then anycable-go would asynchronously broadcast the message out to listeners. What am I missing?

3 replies

Ben Chiciudean

@benydc

is it possible to see the total number of channels connected?

1 reply

Vladimir Dementyev

@palkan

What do you exactly mean by channels?

Charles Boulengé

@Shankhara

Hi everyone,
First I'd like to thank EvilMartians for their incredible work.
I'm working on a project with AnyCable on Rails and a Apollo Client.
I've only used Queries and Mutations, I am able to use Subscriptions without a PRO version ?
Thx

4 replies

Vladimir Dementyev

@palkan

[Help needed] Health Checks for anycable RPC on ALB target groups: https://github.com/anycable/anycable/discussions/140

F Guerra

@tijuanate_twitter

HI,
AnyCable is an awesome tool, and EvilMartians does a great job with it.
We have running any-cable go and any-cable rails on a couple of web apps,
A couple of weeks ago we start seeing memory issues on the servers where the any-cable rails clients run,
after some research we can see the any-cable process building up memory little by little.
There are some tickets about Google ProtoBuf as a suspect of this, but we upgraded to the version that supposedly fixes the issue, and we still see this memory build up happening on idle servers.

Chad

@El-Potato-Slayer

Hi, everyone

I'm in need of some help.

I just migrated to AnyCable from ActionCable. AnyCable-Go and the RPC servers work without any issues. However, once I run the rails server, I get the following message
Successfully upgraded to WebSocket (REQUEST_METHOD: GET, HTTP_CONNECTION: keep-alive, Upgrade, HTTP_UPGRADE: websocket) [2022-05-08 02:24:31.604] ERROR -- NotImplementedError: nil

The error stems from the subscribed method:

def subscribed
    stream_from 'room_channel'
    ActionCable.server.broadcast('room_channel', 'Message sent from backend')
end

Once I remove the stream_from 'room_channel' line in the subscribed method, the error disappears and the front-end is then able to successfully connect to the channel.

Does anyone see what the issue is in the subscribed method?

Here's the link to the repo if anyone wants to take a look

6 replies

Adrien Montfort

@such

Hi,
A gql subscriber was able to DDOS us accidentally because his subscription name was including his channel id and he was subscribing in a loop. This created thousands of different fingerprints for the same subscription which it very long to trigger. Is there a need to include the subscription name in the fingerprint? (using anycable-rails 1.3 and graphql-anycable 1.1.3)

3 replies

Vladimir Dementyev

@palkan

Hey everyone,
We're sunsetting this Gitter channel in favour if GitHub Discussions: https://github.com/orgs/anycable/discussions.
Please, don't hesitate to re-open all the unresolved questions there.

Where communities thrive

People

Repo info

Activity