monix/monix-connect

Pau Alarcón

@paualarco

just released 0.1.0 which is the initial good one, last one was basically testing release process

shall I let people know by sharing it in the main monix channel?

Piotr Gawryś

@Avasil

Of course!

Pau Alarcón

@paualarco

cool :)

Piotr Gawryś

@Avasil

Do you work on it full time?

Pau Alarcón

@paualarco

no haha, I have my full time job but since the isolation I am WFH, so there is no time lost on public transport :P

I used to spend almost 2h every day on the bus and travelling twice a month

Piotr Gawryś

@Avasil

That's rough, I was usually discarding companies with a bad commute times but that's probably hundreds time easier in Cracow than London

Pau Alarcón

@paualarco

you did well then, originally I am from Spain, but sometimes you can't find what you are looking for in your area and you have to go overseas

TapanVaishnav

@TapanVaishnav

Hello all, thank you accepting me on this channel.
I am currently working on good-first-issues.
Also, I like to create a monix-connect for elasticsearch and flink, what do you guys think of the idea?

Pau Alarcón

@paualarco

Welcome @TapanVaishnav :-)
Having those would be amazing! Specially the elasticsearch one, since many companies use it nowadays.

Pau Alarcón

@paualarco

@/all pipeline is back to normal! :D

Pau Alarcón

@paualarco

the reason was that the localstack image was set to latest and they released a breaking changes (but only for some machines) apparently the one that runs our pipeline on github actions was affected.

Alexandru Nedelcu

@alexandru

@paualarco I accidentally pushed a commit: https://github.com/monix/monix-connect/blob/master/website/siteConfig.js

Actually it's this one: monix/monix-connect@eaf733e

It was meant to be a PR, but apparently I'm bad at using GitHub's interface.

Pau Alarcón

@paualarco

@alexandru no problem, unfortunately domain changes haven't applied

will roll it back :P

Alexandru Nedelcu

@alexandru

@paualarco I think it would need to finish the build or something

In the settings file, it gives a warning that the CNAME file doesn't have a correct format: https://github.com/monix/monix-connect/settings

This means the GitHub Pages haven't been deployed yet (an assumption).

Pau Alarcón

@paualarco

Mmh okay, let's give it some time then, fingers crossed 🤞

Alexandru Nedelcu

@alexandru

Are you fine with the domain change? I was meaning to ask in the PR 😅

I mean it's a proposal. If you don't like it you can revert.

Pau Alarcón

@paualarco

I like it! Thanks for helping with this :)

Alexandru Nedelcu

@alexandru

It is live: https://connect.monix.io/

API link is broken: https://connect.monix.io/api/monix/connect/index.html

Is the API published along with the website? As I'm not seeing it in gh-pages: https://github.com/monix/monix-connect/tree/gh-pages

Pau Alarcón

@paualarco

nice!

no, api docs are not there yet monix/monix-connect#149

Piotr Gawryś

@Avasil

What do you think about creating a new module in monix-connect for I/O operations, something like akka-stream-alpakka-file or fs2 io? There's a contributor that is implementing gzip for Observable for STTP and we're wondering where to put it :)

Pau Alarcón

@paualarco

@Avasil sure! the google-cloud-storage connector is already exposing some methods for reading and writing from files which we could abstract in a different module

Pau Alarcón

@paualarco

though it might overlap a bit with monix-nio

Piotr Gawryś

@Avasil

@ghostbuster91 raised a good point to me that compression is not necessarily about files, or I/O so I think we could add it to monix-reactive, fs2 does something similar (text package is in core)

Pau Alarcón

@paualarco

yup, that's right :)

Pau Alarcón

@paualarco

have created monix/monix-connect#321 for bringing downloadMultipart, existsBucket, existsObject, copyObject and an enhanced listObjects operations, will leave it there for a couple of days :)

Tyler M. Kontra

@ttymck

Hi @paualarco just wanted to drop in to say thanks for answering my questions so far, and I hope I can help out with this project :smile:

Pau Alarcón

@paualarco

@ttymck no problem, thank you too for helping :)

Thijs Broersen

@ThijsBroersen

I found an issue in S3 adapter and thought I could make an easy fix and do a PR but I cannot get started, aws-auth does not compile ... getting

[error] ## Exception when compiling 6 sources to /home/thijs/repositories/monix-connect/aws-auth/target/scala-2.13/classes
[error] java.lang.StackOverflowError
[error] scala.tools.nsc.typechecker.Typers$Typer.typedBlock(Typers.scala:2571)
[error] scala.tools.nsc.typechecker.Typers$Typer.typedOutsidePatternMode$1(Typers.scala:5911)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5946)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5993)
[error] scala.tools.nsc.typechecker.Typers$Typer.typedTyped$1(Typers.scala:5703)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5939)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5993)
[error] scala.tools.nsc.typechecker.Macros$DefMacroExpander.$anonfun$onSuccess$1(Macros.scala:631)
[error] scala.tools.nsc.typechecker.Macros$DefMacroExpander.typecheck$1(Macros.scala:631)
[error] scala.tools.nsc.typechecker.Macros$DefMacroExpander.onSuccess(Macros.scala:643)
[error] scala.tools.nsc.typechecker.Macros$MacroExpander.$anonfun$expand$1(Macros.scala:582)
[error] scala.tools.nsc.Global.withInfoLevel(Global.scala:228)

wanted to adjust this:

def deleteObject(
    bucket: String,
    key: String,
    bypassGovernanceRetention: Option[Boolean] = None,
    mfa: Option[String] = None,
    requestPayer: Option[String] = None,
    versionId: Option[String] = None)(implicit s3AsyncClient: S3AsyncClient): Task[DeleteObjectResponse] = {
    val request: DeleteObjectRequest =
      S3RequestBuilder.deleteObject(bucket, key, bypassGovernanceRetention, mfa, requestPayer, versionId)
    S3RequestBuilder.deleteObject(bucket, key, bypassGovernanceRetention, mfa, requestPayer, versionId)
    deleteObject(request)
  }

into

def deleteObject(
    bucket: String,
    key: String,
    bypassGovernanceRetention: Option[Boolean] = None,
    mfa: Option[String] = None,
    requestPayer: Option[String] = None,
    versionId: Option[String] = None): Task[DeleteObjectResponse] = {
    val request: DeleteObjectRequest =
      S3RequestBuilder.deleteObject(bucket, key, bypassGovernanceRetention, mfa, requestPayer, versionId)
    deleteObject(request)
  }

Pau Alarcón

@paualarco

@ThijsBroersen thanks for reporting that! I realised about that last week but didn't changed yet... though it does not alter the functionality of the method

Pau Alarcón

@paualarco

compiling that submodule should be fine, maybe that's something to do with your local config? have seen a similar where it seems like they fixed it by increasing -Xss8m.

Thijs Broersen

@ThijsBroersen

I already did s3/compile ... did not work.

regarding the change ... notice that I also removed the implicit s3-client as it is already part of the S3 trait and was not used by the method implementation

Pau Alarcón

@paualarco

have you changed the SBT_OPS before?

maybe first try just compiling the aws-auth subproject?

sbt -mem 2048 aws-auth/compile

Thijs Broersen

@ThijsBroersen

that works, thanks!

didn't find this tip in the monix-connect documentation :P

Pau Alarcón

@paualarco

no problem, that's a handy one :)

Where communities thrive

People

Repo info

Activity