These are chat archives for petabridge/akka-bootcamp

25th
Sep 2015
Bobby
@bobbychopra
Sep 25 2015 04:10
@Aaronontheweb Using Git Shell (Powershell based), renaming folder didn't identify/push local changes. On my machine, folder is "Images" (renamed it) but on github it is "images." Now all is fixed
Bobby
@bobbychopra
Sep 25 2015 04:33
@raymens thank you. Just trying to contribute in any way I can, working from documentation, then may move to helping the team out with some open code items
richtobey
@richtobey
Sep 25 2015 23:11
Aaron, from this email you sent about new .net stack ... How do you access Spark from .net since there is no interface?
Aaron Stannard
@Aaronontheweb
Sep 25 2015 23:12
@richtobey Hi Rich! I do it from Cassandra
each time I write application data to a Cassandra SSTable it can be picked up and streamed by Spark
can do it with the OSS version of Cassandra or DataStax Enterprise
as for where Spark "outputs" to
I haven't tried having it write to SQL Server
richtobey
@richtobey
Sep 25 2015 23:13
yeah, I'm aware of the datastax driver. haven't played with it yet though.
Aaron Stannard
@Aaronontheweb
Sep 25 2015 23:13
but that's on my to do-list, as that'll open the gates wide open for .NET developers
I talked with @LukeTillman at the Cassandra Summit yesterday about writing a .NET driver for Spark's DataFrame API
so that's another option
richtobey
@richtobey
Sep 25 2015 23:13
thumbs up.
Aaron Stannard
@Aaronontheweb
Sep 25 2015 23:14
that's at the "idea was discussed over beers" stage though
just to set expectations :p
richtobey
@richtobey
Sep 25 2015 23:14
right. got it. thx for quick reply.
Aaron Stannard
@Aaronontheweb
Sep 25 2015 23:14
anytime man - glad you asked!
richtobey
@richtobey
Sep 25 2015 23:14
have you looked at filodb? looks very related.
Aaron Stannard
@Aaronontheweb
Sep 25 2015 23:14
I haven't! Got a link for it?
richtobey
@richtobey
Sep 25 2015 23:15
yeah. one sec.
uses tachyon and kafka in addition to cassandra and spark for real time analytics. would be great to get access from .net. it may be you can do it through cassandra as you mentioned. I'm just starting to scratch the surface on it.
Aaron Stannard
@Aaronontheweb
Sep 25 2015 23:18
I'm going to have to check this out - looks right up my alley
richtobey
@richtobey
Sep 25 2015 23:18
yeah, i thought so.
Aaron Stannard
@Aaronontheweb
Sep 25 2015 23:18
I saw a talk some guys from British Gas gave on using cassandra + spark for real-time machine learning stuff with IoT for smart meters
basically being able to text someone and let them know they left their oven on as they were leaving home
richtobey
@richtobey
Sep 25 2015 23:19
I started to learn scala as part of understanding it, but it's an ass kick. will take a hot second.
Aaron Stannard
@Aaronontheweb
Sep 25 2015 23:19
made me think I should be playing with Spark a lot more than I have
I used to use Hive for this stuff, even last year
richtobey
@richtobey
Sep 25 2015 23:20
I'm probably reinventing the wheel. anyway, loved your email on new stack. was headed in that direction, trying to refresh some skilz. couldn't understand the spark piece b/c of lack of interface.
yeah, it's changing quickly. my head is spinning.
Aaron Stannard
@Aaronontheweb
Sep 25 2015 23:20
totally get it - glad you liked it. And yeah, I guess that point wasn't very clear
richtobey
@richtobey
Sep 25 2015 23:21
no worries. I'll look forward to your email on adding tachyon and kafka into the mix.
;-0
Aaron Stannard
@Aaronontheweb
Sep 25 2015 23:23
haha