Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Scott Mansfield
    @ScottMansfield
    I would suggest that you create an EVCacheClient instance and inject that instead
    Gang
    @shengang1978
    wow. The EVCacheClient, I think, is too low level interface. To use it, I need dig into and study more details. Actually, I just want to use the EVCache in my project in current stage as soon as possible. It seems the Inject is not an easy way to me now. And will you abandon the way of creating EVCache.Builder in future? Or will you have an Inject sample soon?
    Scott Mansfield
    @ScottMansfield
    The Builder is used and supported right now. Let me look for an example of proper init
    Gang
    @shengang1978
    :) This is good news to me. I can use the Builder as the sample first. I can update after your example.
    Scott Mansfield
    @ScottMansfield
    if the Builder works for you, e.g. you can verify it's calling out to memcached, then you can safely ignore that warning log message
    we've already updated the message to be more explicit that it can be ignored based on how you're using the client
    Gang
    @shengang1978
    Yes, the Builder works even the warning log. I'll do more test and use it. BTW, do you plan to support the group of the cache keys? In our project, I have to clear much cache data in one operation. If we can put the cache keys in one group, it's easy to clear the cache.
    Scott Mansfield
    @ScottMansfield
    memcached does not have a concept of grouping different keys together
    and the client will not keep track. If you want to invalidate a group typically you have to store it somewhere else
    for example, there's a team internally that uses an index key to access the other keys
    if you write the keys that you want grouped as a value for a group key, you can read it back later and go invalidate the keys as a group
    it won't be atomic
    but that's normally just fine
    Either way, let me know if you have any other questions on it. We're in California so if we're not replying it might just be early morning.
    Scott Mansfield
    @ScottMansfield
    and we can help a little with some architecture, as we've seen a bunch of different use cases internally
    if you want I can send you a video from Strange Loop where we described some of them
    Gang
    @shengang1978
    Thanks. Actually, I meet a problem of clearing the cache. Some operations need to clear many related cache data. The problem is that when we add new features which are related these operations, we forget to add the logic of clearing, updating the cache data. Only when our clients say "there is wrong in our App", then we find we forget to clear the cache.
    Now, the cache layer design is very complicated. I want to redesign it. But I have no idea now. It seems there isn't a easy to avoid my problem.
    Scott Mansfield
    @ScottMansfield
    that is an interesting problem, it almost sounds like you actually need indexes and foreign keys... maybe a RDBMS would better fit?
    i don't have too much detail to go on, but it sounds like there's a layer of relationships on top of memcached, which you can either store as separate data in memcached or in a secondary index somewhere else
    if you have simple groupings, you can store a group key where the value is all of the keys in the group
    Gang
    @shengang1978
    Yes, this is the basic idea. I want to build the secondary index. But these indexes are related to business. It means I have to add the index to every business. Key group cannot resolve this problem. For example, in IM, we have a group persons. After adding/deleting aperson to/from group, the cache of the group persons has to be cleared. Then we add a new operation: update the user profile. It needs to clear the cache too. But we forget to add the clearing cache after the updating. ... And there are too many similar business logic.
    Do you understand my problem?
    Scott Mansfield
    @ScottMansfield
    I do a little bit better now, thank you. It sounds like a hard problem to solve
    Really, this is what triggers on a database are for
    I'm thinking more about this
    Gang
    @shengang1978
    Thanks for your time. I'll think more about the problem. Your idea RDBMS may be a good idea. Let me think to build the index in the database.
    Scott Mansfield
    @ScottMansfield
    related to forgetting to invalidate: unfortunately that's the bane of everyone's existence. Cache invalidation is one of the hard problems. Basically it's better testing and better context for the people developing
    normally, the easiest way to manage a single thing, like IM groups, would be to make a service to manage them
    you can either have a client that goes directly to the cache and then the service
    or have a plain REST service with the cache behind
    in that way, all management of groups is done in one place and every time that service gets an update it can invalidate the cache
    Gang
    @shengang1978
    The EVCache uses the system properties. It may be easy to configure. But it makes trouble when I do the key group feature. I want to use the EVCache in memory cache and I want to read/write the memcached cache without the in memory cache. I don't find a way in EVCache which let me use both.
    Now, I have to use the EVCache + In memory cache. And use the spymemcached client for the key group.
    Gang
    @shengang1978
    And I still find two traps. 1. The AppName (prefix of the settings. ig. The "EVCACHE" of the property EVCACHE..use.simple.node.list.provider=true) must be capital letters. Otherwise, the EVCache doesn't work. 2. If you load the properties from a file by the ConfiurationManager.loadPropertiesFromResources(...), the EVCache won't use the default settings anymore.
    And there is another strange log: 2016-10-26 22:50:10.779 INFO com.netflix.evcache.EVCacheTranscoder: Compression increased the size of infrastructure.MMData from 160 to 169
    Does EVCache compress the data? But it seems if the data is small, the compressed data is bigger than the original data. Is it the log wrong?
    Gang
    @shengang1978
    Another question about the serialize. In my project, I use the POJO classes which are not inherit from the Serializable interface. It's OK because we use the gson and cxf json provider. But I find it's a trouble to EVCache. It cannot serialize these classes. It's really annoying. I have two choices: 1. Update all the POJO classes to implement the Serializable interface. 2. Use the protobuf to serialize the classes to byte array.
    About the Key group feature, you mentioned that your internal team uses the same idea. How do they resolve the problem that the Builder cannot use the different set of the settings?
    Scott Mansfield
    @ScottMansfield
    The in-memory cache is meant to relieve pressure on remote caches when the request volume is very high for only a few of keys. Our majority use case does not include it but some internal customers of ours do run with it on in production
    it shouldn't affect the writes to the remote cache
    The properties are case sensitive, which is just the way they are designed.
    The default settings are per-property. If you load new values for all of the properties you will end up overwriting them... I think I'm not quite understanding what you mean about the default settings.
    the internal team that uses a group of keys has application level logic to write a group key with the other keys as the value
    so the key would be foo_group or foo_index for example
    and the value would be foo1,foo2,foo3 which can be read, split, and used to do operations on the whole group
    Gavin McQuillan
    @gmcquillan
    I was curious about deploying EVCache on bare metal, but looking through the code, I notice in some of the connection pooling classes that there's a bunch of AWS-specific logic.
    How broad is support for environments outside of AWS?
    Also, are there any docs on which CAP tradeoffs are made?
    Desmond Vehar
    @desmondvehar_twitter
    @gmcquillan/future folk There is some details CAP tradeoffs in the wiki: https://github.com/Netflix/EVCache/wiki
    Notably in the some of the deployment configurations they call out that data is sharded so reads hit the same zone every time but writes are replicated across all so zones cache-miss recovery from the local zone can be done more quickly than reading from the original data source.