47deg/fetch

Simple & Efficient data access for Scala and Scala.js http://47deg.github.io/fetch/

Peter Neyens

@peterneyens

A quote from the Haxl paper :

When the business logic is only concerned with reading data
from external sources and not writing, the programmer doesn’t
care about the order in which data accesses happen, since there
are no side-effects that could make the result different when the
order changes. So in this case the programmer would be entirely
happy with not having to specify either ordering or concurrency,
and letting the system perform data access in the most efficient way
possible.

Tristan Lohman

@gatorcse

My use case was in some light read-transform-write batch jobs we run

we have a query for a bunch of IDs, and for each of those we have to read some data, so some work, and then write some data back out. We do the batching manually right now (usually as one big batch or all individual requests) but fetch’s batch/parallelism while letting the functions look like their operating on one entity at a time is pretty appealing

But I guess these are special cases of writes, which are idempotent and do not affect any of the reads

Justin Heyes-Jones

@justinhj

I was also wondering about the utility of a Put style version of Fetch. The interesting part for me would be combining Put and Fetch jobs together and have Put be able to update the cache that the Fetch jobs are using. But as @peterneyens says there would be a lot of edge cases to think about

Raúl Raja Martínez

@raulraja

@gatorcse I still think it's a good idea and if it's explicit enough so that the user understands the behavior it's something we could have in Fetch as a separate companion module

Andrey Ivanov

@a-nigredo

Hi all, I try to use fetch "0.6.3" with cats "1.0.0-MF" and in run-time I see an exception

Exception in thread "main" java.lang.NoClassDefFoundError: cats/FunctorFilter

What I missed?

This is my dependencies:

libraryDependencies += "org.typelevel" %% "cats-core" % "1.0.0-MF"
libraryDependencies += "com.47deg" %% "fetch" % "0.6.3"

Raúl Raja Martínez

@raulraja

@a-nigredo Fetch still depends on cats 0.9.0 which had the FunctorFilter there. We are currently working towards moving it to 1.0.0-MF and FunctorFilter has moved to FunctorEmpty in a new lib cats-mtl. @peterneyens do you know how close are we to release Fetch for cats 1.0.0-MF?

Peter Neyens

@peterneyens

We would need a Monix release using cats 1.0.0-MF.

Raúl Raja Martínez

@raulraja

@peterneyens Looks like Alex recently addressed that https://github.com/monix/monix/releases/tag/v3.0.0-M1

eranmiz132

@eranmiz132

hi guys

i am new to fetch, and i will be glad if someone can help a few issues:

can i create an api which is not part of the DataSource trait, and call it ?

can i define an api which save/update and not just fetching data ?

Peter Neyens

@peterneyens

@eranmiz132
The only way Fetch can reach out to external sources is through the fetchOne and fetchMany methods of DataSource.
Can you give an example what you want to accomplish?

For the second point, there was some discussion in the posts above about just that (starting at https://gitter.im/47deg/fetch?at=599de37042d2f61025897618).
Summary: For the moment Fetch only supports fetching data

eranmiz132

@eranmiz132

thank u for the reply ! well, i have a lot of different api which get items, but the for different keys. if i extends the DataSource trait, i can implement only one "kind" of get by key, or i need a lot of classes to extends the DataSource, each with different key. it is not convenient.

chris vale

@crispywalrus

@peterneyens I opened a (trivial) PR against your 1mf branch that changes the name of the twitter module from twitter to fetch-twitter

the branch itself seems in pretty good shape, I was able to upgrade code without incendent

Peter Neyens

@peterneyens

I missed that it was for my branch. Thanks for the new PR anyway.

chris vale

@crispywalrus

@peterneyens np!

Peter Neyens

@peterneyens

@/all We just released Fetch v0.7.0 https://twitter.com/47deg/status/916369464161128449

Raúl Raja Martínez

@raulraja

:clap:

Justin Heyes-Jones

@justinhj

Hi! I made this side project while I was on vacation I was inspired by your issue #11 (visualize fetch rounds with reftree). There is a blog post and github repo http://justinhj.github.io/2017/10/11/hacker-news-api-3.html

Peter Neyens

@peterneyens

@justinhj Nice!

The reftree visualization looks quite nice indeed.

Justin Heyes-Jones

@justinhj

Yeah it's a neat library

Raúl Raja Martínez

@raulraja

@justinhj awesome post and project!

Justin Heyes-Jones

@justinhj

Thanks!

Taleb Zeghmi

@talebzeghmi

Does anyone use the Fetch API in production?

Tom Adams

@tomjadams

Yes. Have done for months.

Taleb Zeghmi

@talebzeghmi

Has anybody found a Redis Scala API that supports Cluster pipelining?

Justin Heyes-Jones

@justinhj

I'm using https://github.com/etaty/rediscala which should in theory but I don't know if it allows pipelining what you want specifically as I haven't used the cluster support yet.

Taleb Zeghmi

@talebzeghmi

rediscala purports to support pipeline but uses transactions

Aarsh Shah

@aarshkshah1992

Hello Guys

I am Aarsh , a Scala dev looking to contribute to some open source projetcs

Is it possible for me to work on Fetch

Considering that it's been developed by 47 degrees ?

Raúl Raja Martínez

@raulraja

@aarshkshah1992 of course! your contributions will be very welcomed!

Aarsh Shah

@aarshkshah1992

Thanks for the reply . Is there a contributing guide I can look into ?

Also, any list of planned features / issues/ bug fixes I can work on to get my foot in the door ? Would really appreciate the help.

Raúl Raja Martínez

@raulraja

no guide at the moment since it's a self contained and small library, feel free to glance over the issues and pick up any of the current open ones and ask any questions here :)

Aarsh Shah

@aarshkshah1992

Great, thanks a lot :smile:

Raúl Raja Martínez

@raulraja

:thumbsup:

Tom Adams

@tomjadams

Silly question, has anyone tried to use fetch from java? Or, know a similar library that’s usable from java?

kerr

@hepin1989

@tomjadams You will need to wrote a wrapper around it

Tom Adams

@tomjadams

@hepin1989 Thanks, that’s what I figured.

Raúl Raja Martínez

@raulraja

Java has no way to represent scala higher kinds and some of the methods would need to be made concrete or specialized. Same for the implicit resolution would have to be wrapped in a scala project that exposes concrete types and already resolved instances with signatures compatible in Java. If you plan on going down that rabbit hole we'd welcome such a submodule in Fetch itself

Where communities thrive

People

Repo info

Activity