Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Aug 06 11:45
    paulpdaniels commented #204
  • Aug 05 04:43
    cbrunnkvist edited #204
  • Aug 05 04:42
    cbrunnkvist opened #204
  • Jul 11 15:34
    calvellido synchronize #203
  • Jul 11 15:34

    calvellido on paolo-201-implement-redesign

    Drop tests support for Oracle J… (compare)

  • Jul 11 14:58
    purrgrammer commented #203
  • Jul 11 13:21
    calvellido commented #203
  • Jul 11 13:13
    calvellido synchronize #203
  • Jul 11 13:13

    calvellido on paolo-201-implement-redesign

    Use Ubuntu Xenial to run Travis… (compare)

  • Jul 11 12:56
    AntonioMateoGomez edited #203
  • Jul 11 12:41
    AntonioMateoGomez synchronize #203
  • Jul 11 12:41

    AntonioMateoGomez on paolo-201-implement-redesign

    Delete sass-cache (compare)

  • Jul 11 12:25
    AntonioMateoGomez ready_for_review #203
  • Jul 11 12:25
    AntonioMateoGomez synchronize #203
  • Jul 11 12:25

    AntonioMateoGomez on paolo-201-implement-redesign

    Modifies sidebar links in doc s… (compare)

  • Jul 11 11:19
    israelperezglez synchronize #203
  • Jul 11 11:19

    israelperezglez on paolo-201-implement-redesign

    fix sidebar docs (compare)

  • Jul 11 10:30
    israelperezglez synchronize #203
  • Jul 11 10:30

    israelperezglez on paolo-201-implement-redesign

    review styles (compare)

  • Jul 10 13:00
    AntonioMateoGomez synchronize #203
Srepfler Srdan
@schrepfler
sweet
thanks Tom
Tom Adams
@tomjadams
👍🏼
Raúl Raja Martínez
@raulraja
@/all Fetch 0.6.3 released https://twitter.com/47deg/status/900076119000571904 Thanks everyone for your contributions!
Justin Heyes-Jones
@justinhj
👍🏼
Tom Adams
@tomjadams
👍🏼
Tristan Lohman
@gatorcse
Is there similar library to this for posting data?
Just looking at the API, could this be used to send/store data as well as fetch it?
DataSource[I, A], with I as your object type and A as your datasource api result type (DBResult or whatever)
apologies if this has been asked before
Raúl Raja Martínez
@raulraja
@gatorcse I think the value of Fetch is in it's optimizations for deduplication, batching and parallelism. If we are talking about posting data and using something like Fetch or Fetch itself we would have to determine what those mean in the context of posting.
  • Can we batch posts? that is probably doable in the same way we use datasources today
  • Can we deduplicate posts? Maybe if once one is acknowledged posts with the same data to the same id are ignored.
  • Can we paralelize posts? As long as the semantics are of independent operations and the value does not matter and the order in which they happen does not affect the result of the program then yes.
    So, yes, something for posting data could potentially be implemented as part of fetch or companion lib and would allow users not to worry about keeping track of the ops they perform to manually not do them again, batch them explicitly or run them in parallel.
Peter Neyens
@peterneyens

The benefits aren't as big as when fetching data I think.

The main benefit with Fetch (and other libraries inspired by the Haxl paper) is that you can just specify the data you need to use without thinking how to get it (efficiently).
If part A, part B and part C all independently need foo you don't need to think about where to get foo so you only request it once.
If parts A, B and Care combined in one Fetch[Result], foo will automatically be fetched only once.

Posting data can also be a lot more complex because posts can influence each other.
For example what do you do with two parallel posts with the same id but different data?

I am interested to know if you have a use case in mind or if it is just a general question.

A quote from the Haxl paper :

When the business logic is only concerned with reading data
from external sources and not writing, the programmer doesn’t
care about the order in which data accesses happen, since there
are no side-effects that could make the result different when the
order changes. So in this case the programmer would be entirely
happy with not having to specify either ordering or concurrency,
and letting the system perform data access in the most efficient way
possible.

Tristan Lohman
@gatorcse
My use case was in some light read-transform-write batch jobs we run
we have a query for a bunch of IDs, and for each of those we have to read some data, so some work, and then write some data back out. We do the batching manually right now (usually as one big batch or all individual requests) but fetch’s batch/parallelism while letting the functions look like their operating on one entity at a time is pretty appealing
But I guess these are special cases of writes, which are idempotent and do not affect any of the reads
Justin Heyes-Jones
@justinhj
I was also wondering about the utility of a Put style version of Fetch. The interesting part for me would be combining Put and Fetch jobs together and have Put be able to update the cache that the Fetch jobs are using. But as @peterneyens says there would be a lot of edge cases to think about
Raúl Raja Martínez
@raulraja
@gatorcse I still think it's a good idea and if it's explicit enough so that the user understands the behavior it's something we could have in Fetch as a separate companion module
Andrey Ivanov
@a-nigredo
Hi all, I try to use fetch "0.6.3" with cats "1.0.0-MF" and in run-time I see an exception
Exception in thread "main" java.lang.NoClassDefFoundError: cats/FunctorFilter
What I missed?
This is my dependencies:
libraryDependencies += "org.typelevel" %% "cats-core" % "1.0.0-MF"
libraryDependencies += "com.47deg" %% "fetch" % "0.6.3"
Raúl Raja Martínez
@raulraja
@a-nigredo Fetch still depends on cats 0.9.0 which had the FunctorFilter there. We are currently working towards moving it to 1.0.0-MF and FunctorFilter has moved to FunctorEmpty in a new lib cats-mtl. @peterneyens do you know how close are we to release Fetch for cats 1.0.0-MF?
Peter Neyens
@peterneyens
We would need a Monix release using cats 1.0.0-MF.
Raúl Raja Martínez
@raulraja
@peterneyens Looks like Alex recently addressed that https://github.com/monix/monix/releases/tag/v3.0.0-M1
eranmiz132
@eranmiz132
hi guys
i am new to fetch, and i will be glad if someone can help a few issues:
  1. can i create an api which is not part of the DataSource trait, and call it ?
  1. can i define an api which save/update and not just fetching data ?
Peter Neyens
@peterneyens

@eranmiz132
The only way Fetch can reach out to external sources is through the fetchOne and fetchMany methods of DataSource.
Can you give an example what you want to accomplish?

For the second point, there was some discussion in the posts above about just that (starting at https://gitter.im/47deg/fetch?at=599de37042d2f61025897618).
Summary: For the moment Fetch only supports fetching data

eranmiz132
@eranmiz132
thank u for the reply ! well, i have a lot of different api which get items, but the for different keys. if i extends the DataSource trait, i can implement only one "kind" of get by key, or i need a lot of classes to extends the DataSource, each with different key. it is not convenient.
chris vale
@crispywalrus
@peterneyens I opened a (trivial) PR against your 1mf branch that changes the name of the twitter module from twitter to fetch-twitter
the branch itself seems in pretty good shape, I was able to upgrade code without incendent
Peter Neyens
@peterneyens
I missed that it was for my branch. Thanks for the new PR anyway.
chris vale
@crispywalrus
@peterneyens np!
Peter Neyens
@peterneyens
@/all We just released Fetch v0.7.0 https://twitter.com/47deg/status/916369464161128449
Raúl Raja Martínez
@raulraja
:clap:
Justin Heyes-Jones
@justinhj
Hi! I made this side project while I was on vacation I was inspired by your issue #11 (visualize fetch rounds with reftree). There is a blog post and github repo http://justinhj.github.io/2017/10/11/hacker-news-api-3.html
Peter Neyens
@peterneyens
@justinhj Nice!
The reftree visualization looks quite nice indeed.
Justin Heyes-Jones
@justinhj
Yeah it's a neat library
Raúl Raja Martínez
@raulraja
@justinhj awesome post and project!
Justin Heyes-Jones
@justinhj
Thanks!
Taleb Zeghmi
@talebzeghmi
Does anyone use the Fetch API in production?
Tom Adams
@tomjadams
Yes. Have done for months.
Taleb Zeghmi
@talebzeghmi
Has anybody found a Redis Scala API that supports Cluster pipelining?
Justin Heyes-Jones
@justinhj
I'm using https://github.com/etaty/rediscala which should in theory but I don't know if it allows pipelining what you want specifically as I haven't used the cluster support yet.
Taleb Zeghmi
@talebzeghmi
rediscala purports to support pipeline but uses transactions
Aarsh Shah
@aarshkshah1992
Hello Guys
I am Aarsh , a Scala dev looking to contribute to some open source projetcs