Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jan 06 2016 17:57
    test2
  • Jan 06 2016 17:56
    test
Adi
@porta_adi_twitter
so, basically the question is how can I get entire documents content having a view that doesn't export entire document content (as is not a best practice to export it entirely) and do it in a single operation:
for(ViewRow row : result.allRows()) {
row.valueAs(JsonObject.class) -> is the view Output, not the entire document
}
TheHunter
@TheHunter
Hi guys
I'd like to share a question about net SDK, Is this the right place to share a question ?
Brant Burnett
@brantburnett
@TheHunter sure
TheHunter
@TheHunter
Hi @brantburnett :
thanks for your availability, so I try to explain the problem that I've found in NET SDK, currently It occurs the Query service is removed by this logic (class ClusterContext, method ProcessClusterMapAsync, row number. 456):
image.png
TheHunter
@TheHunter
When the condition is true, the local variable nodeAdapter sets the bucketNode.NodesAdapter property (row 459), this causes an error because the clusterContext has the same instance of IClusterNodes, so when the condition is true (row 456) changes completely the behaviour of the retrieved node (that is shared between a Bucket and ClusterContext)
the side effect is, that ClusterNode is overridden, and I lose some references like flag for query service.
This problem occurs when I create a bucket in my code, otherwise it works !
TheHunter
@TheHunter
the given example goes in error
image.png
and this example is similar to the previous:
image.png
But in this last case, I don't create any Bucket instance, and so It works !!!
TheHunter
@TheHunter
there's no problem with connections or something like that, with my test I could verify that creating a bucket behind the scenes the code doesn't work well, in my case the Query service mapped in IClusterNode is overridden, so loosing the flag HasQuery (to false).. so no service is found and any kind of query couldn't execute.
Brant Burnett
@brantburnett
@TheHunter so, to make sure I understand, the problem is that N1QL queries can fail if you attempt to open a bucket that doesn't exist before you run the query? Or is it a bucket that does exist?
Also, are you on SDK 3.1.1 that was released last week?
TheHunter
@TheHunter

@brantburnett , yes exactly.. the real problem occurs after this:

IBucket bucket = await cluster.BucketAsync("semantic");

after the previous instruction, any query fails, example:
cluster.QueryAsync<dynamic>($"select d.* from {bucketName} d limit 1")

The bucket on my cluster exists naturally

Just today morning I've merged my cloned repository with the current official master, and the problem is still there.
Brant Burnett
@brantburnett
What happens if you add a cluster.WaitUntilReadyAsync call? @TheHunter
TheHunter
@TheHunter
I can try now.. just a moment
TheHunter
@TheHunter
nothing changes @brantburnett
In my opinion, the problem is on the class ClusterContext, method ProcessClusterMapAsync()
image.png
If a bypass that "if", everything works!
so, the problem is when the condition is true, because the IClusterNode found (var bucketNode) is changed by the current variable "nodeAdapter" , when:
bucketNode.NodesAdapter = nodeAdapter;
Brant Burnett
@brantburnett
@TheHunter I can't be certain from the snippets you provided, but I believe your issue may be related to how you're disposing or bootstrapping the cluster. I haven't been able to replicate it. Can you provide more details about what's behind GetClusterAsync from your snippet? Or, better yet, provide a full reproduction on GitHub?
TheHunter
@TheHunter
Hi @brantburnett ,
I can share my unit test, applied directly on the current SDK project test
the problem is not the bootstrapping the cluster., but I'd try to push on github my snipped code ..
TheHunter
@TheHunter
Hi @brantburnett :
this is my unit test on my github profile:
https://gist.github.com/TheHunter/027ec89a678b6c23934c59f6e38e51c4
Brant Burnett
@brantburnett
@TheHunter that looks fine, generally matches how I'm trying to reproduce. What's your cluster topology for your test? That could be the difference.
TheHunter
@TheHunter
@brantburnett :
sure, but I'm wondering how come in one case always works, but in the other case not.
The strange thing is when I create the Bucket ( method BadQueryAsyncTestAsync()) the problem occurs, instead in the first case ( method GoodQueryAsyncTestAsync) works always
in any case, my cluster topology is this.
image.png
TheHunter
@TheHunter

The cluster is deployed in AKS (azure), done by kubernetes-operator using HELM chart

The cluster is composed by:

  • 3 servers (PODs) for DATA & INDEX services together
  • 1 server (POD) for QUERY service
  • 1 server (POD) for SEARCH service

Everything deployed in the same NODE belong to a POOL.

Being a development environment, It's not the case to deploy into different NODES

There's no a network issue, the UI console works, and the unit test (the method which It doesn't use the bucket) works always.
Brant Burnett
@brantburnett
@TheHunter can you talk to me about that Nginx ingress controller you're showing between your test and the cluster? This appears to me to be a nonstandard approach and may be related to the problems.
TheHunter
@TheHunter

@brantburnett :
Sure, I can talk about my nginx controller, this is a common way to expose any kind of service outside the cluster on kubernetes, but consider that the problem occurs even inside the cluster, so
my POD Reader runs inside the kubernetes cluster and It has the problem I've mentioned before.

Inside the cluster I use the {cluster-name}-srv service directly as explained by the documentation, outside the cluster, by way of Nginx the {cluster-name}-srv service is exposed.

this is the configuration of the Load balancer used by Ingress controller
image.png
as you can see, the port 11210 is exposed
TheHunter
@TheHunter
In any case, if the problem was network, the problem would be present always, and not for a single particular case, would you be agreed with this opinion ?
I'd be happy if the problem would be NGINX, in that case my PODs running inside the cluster should run correctly ..
Brant Burnett
@brantburnett
@TheHunter Note that you can't just expose port 11210 on an Nginx proxy externally and have it work. The Couchbase SDK needs port 11210 connections to every node in the cluster, not load balanced. It establishes long running connections to each node in the cluster and routes traffic to the correct node for any given request. If you need to access your cluster from outside Kubernetes, there are features of the Autonomous Operator which help enable that. https://blog.couchbase.com/autonomous-operator-1-2-0-public-connectivity/
that said, I agree that if it happens from inside your cluster it probably isn't the problem. At this point I think it may be related to MDS, I'm testing on a single node locally. Let me try a more complicated cluster and see if I can reproduce it
TheHunter
@TheHunter

@brantburnett : that's right, my current Ingress Controller that uses the above LoadBalancer internally exposes other enterprise services like Redis, RabbitMQ MongoDB etc.
But as I've mentioned before, the problem is not NGINX, because the problem occurs inside kubernetes cluster too, I do the same test inside my POD deployed in the kubernetes cluster.
Yes about not balanced request, in fact the current SDK not do that, the current SDK using the "srv" service makes a "service discovery" for all NodePort services, installed by couchbase-operator (helm chart).
Net SDK at the end gets all service endpoints ( NodePort ), and the sdk client uses those, that's all, so Ingress Controller serves only on bootstrap for the first time the ICluster object, initialising internally all NodePort services for operating with DATA, INDEX, QUERY, SEARCH services.

I don't know if I express well this issue, but believe me !!!, testing directly the current SDK I discover the root of problem, if you prefer to discuss better with a call, we can do it, when you prefer.

TheHunter
@TheHunter
I try to deploy a new small cluster using LoadBalancer instead of NodePort , this is the only thing that I didn't do it before, but It would be a little problematic because I would need another balancer dedicated only for couchbase cluster, a thing that I would want to avoid
TheHunter
@TheHunter
I've created a yml file that I use for my current couchbase cluster installation (using helm chart):
https://gist.github.com/TheHunter/1aae94c66cd745b0aeca87588c054bef
Jeffry Morris
@jeffrymorris
Hi @TheHunter I am the .NET SDK lead and see if I can help you. Have you tried recreating this issue with a cluster outside of K8? If you can do so, it would be helpful so we can isolate the issue. FWIW, there have been bugs here in the past, but they have been resolved(https://issues.couchbase.com/browse/NCBC-2475, https://issues.couchbase.com/browse/NCBC-2475, https://issues.couchbase.com/browse/NCBC-2383).
TheHunter
@TheHunter
Hi @jeffrymorris ,
unfortunately I haven't tried to create a cluster outside K8S, It would be so difficult to do that, we have a local Data Center but It implies that I'll need at least 5 VMs... and our target is kubernetes for infrastructure in any case.
I realise that It's so difficult to replicate this issue, but if you want we can talk about in a meeting, in that way I can show you where is the exact issue that I've found on SDK.
I'm an expert on C# coding, and working as a Software architect.. and I think talking with you would be great and easy, and I'm very confidence that we could understand each other on the fly.
TheHunter
@TheHunter
@jeffrymorris : this issue https://issues.couchbase.com/browse/NCBC-2475 is similar to the current problem I'm facing up.