These are chat archives for nrinaudo/kantan.csv

8th
Jul 2017
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 19:57
@ShaneDelmore you're using kantan.csv for serialisation then? Because that's currently, to my knowledge, the only unsafe bit in the entire API. I just wanted to make sure you were aware of that flaw
Shane Delmore
@ShaneDelmore
Jul 08 2017 19:59
Yeah, we have a Seq of case classes representing user records, then medical clinics and patients can generate an export, which is just us fetching the case classes, then running them through Kantan to generate csv text. Have not had any problems with it yet and we run it with lots of randomly generated data using scalacheck but if you know of issues I can take a look to make sure we are not vulnerable.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:00
well, a single one, really: it can throw on write if an IO exception occurs
nothing else in the whole library throws, just write operations
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:01
Hmm....trying to think if that could be an issue? We stream it out directly to tcp, never write a file, so I guess if the tcp connection failed, but then we would catch that as we catch normal disconnects so I think we're safe.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:02
if you deal with exceptions when writing to your socket, you're safe
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:03
If you know anyone who has Type 1 diabetes, there are only two companies that make continuous glucose monitors, the company I work for https://www.dexcom.com and Medtronic. We have a huge portion of the market, so lot of patients and doctors are benefitting from Kantan every time they download their data from us. Thought that would be fun for you to know. We just entered the European market a few months ago but are rapidly expanding over there.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:03
that is so cool
thanks for letting me know, it really does make me happy to learn that it's being used for a good cause!
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:05
It was a fun sprint, when I worked on it I came in on time easy because I had specced far more time to add the feature, then after trialing a couple of libraries Kantan just worked, ended up doing in a day what I had set aside a week for.
:thumbsup:
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:07
yeah if you're in type classes and hassle-free encoding / decoding, like circe does for JSON, I like to think kantan.csv is pretty good
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:07
It's funny, a surprising number of companies try to lock up users data, they show reports but don't just let you download the csv. So we ended up reverse engineering the protocols of about 20 different glucose monitors so we could use them for our reports, but since we also offer an export the nice side effect is you can use our product to get your glucose data as csv even if the manufacturer doesn't support it directly. And it's free to patients, we only charge doctors.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:08
are you sure you should be mentioning this in a public space? It sounds like something I'd definitely do, but might be slightly dodgy from a legal perspective
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:08
It may be you don't hear about it because you don't have a ton of users, but it also may be you don't hear about it much because it just works. I happen to be online and chatty, but all of my coworkers like Kantan, you just would never hear it from any of them because they don't hang out on Gitter.
Ha....nothing that hasn't been cleared by our lawyers.
We worried about it initially but Dexcom has a pretty big legal team and they were alright with it. Although come to think of it, it is possible we will retire the third party device support do to the expense of maintaining it, but I don't recall them thinking we had any reason to be worried on the legal front.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:10
yeah, I sometimes have cool moments. A few weeks ago I was in Stockholm, giving a talk on type classes, as one does, and the host of the meetup came to me afterwards and said "I knew your name sounded familiar. I've just checked, we use your code in production!"
it took a couple of days before I could stop grinning
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:10
Oh man, that's fun. I hope to hear that about my code someday. That reminds me, I need to put a better readme together for imclipitly. I just published it hoping maybe some people would benefit from it and someone asked me the other day "so I looked at the readme....what does it do?". Doh!
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:11
oh, alright. I'd have thought that if they try to lock the data up, they might frown on your freeing it. But if the lawyers cleared it, reverse-engineer away!
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:11
Yeah, I don't know that they actually try to lock it up, I think they just are incredibly cheap (you can pick up these meters for $25 at your local pharmacy) and don't put any effort into user software.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:12
I must admit - even scala clippy take a few minutes to understand. imclipitly kinds of expect you to know what it is before you come looking for it
also, while the name is cool, it's nigh impossible to google. Did you mean implicitly?
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:14
The name is ridiculous, I admit. I'm the first google result though! https://www.google.com/search?q=imclipitly&ie=utf-8&oe=utf-8
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:14
if you type it just right
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:14
Yeah, I knew it was a bad idea when I did it, just couldn't help myself.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:15
if you type inclipitly, for example, you're not even in the results
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:15
When I feel it is more solid I will just have AdamW link to it from the clippy page, or fold it into his project directly.
But works awesome at my work.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:16
I seriously think if you added a few examples, or one of these fancy GIFs that you see nowdays like coursier does, it would help a lot
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:18
Oooh, great idea. Thanks!
Those Gifs are wonderful.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:18
wait - I thought it'd be some sort of database of "you seem to have forgotten to define an implicit Foobar, see URL for more info", but...
looking at the source code, you generate these dynamically?
what kind of dark magic is that?
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:19
Yeah. If you add it to Kantan, then publish, it will add a json file of advice to your resources, then any clippy users get the advice for free.
It's cool, but I haven't figured out how to get other people to realize it's cool yet. Completely answers all of those "what do I import to get x" questions.
It's dark, dark, scalameta magic.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:20
jesus
you realise that a good half of the scala community would kill for something like that?
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:20
No, actually scalameta makes it ridiculously easy to do. If you look at the source there is more to the sbt file than there is to the code generation.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:20
you wrote it, and sort of hid behind a fun name and no documentation?
do you take pleasure in torturing people ?!?
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:21
I mentioned it to a few people, they didn't seem terribly interested. But at work we put it in our utility libraries, nobody ever has to ask what do I import to get .times or whatever.
Ha. I am bad on docs. Completely opposite of your docs.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:21
yeah but I mean add a complete example and use case on your README.md!
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:21
But you have convinced me, I will fix the readme with some cool examples.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:23
if anything, OSS library maintainers will love it because they can answer all the "what do I import..." questions with "install imclipitly"
or, at least, "have you installed imclipitly? what was the suggestion? did you try that? well then?"
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:23
Do you get many of those?
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:23
I personally don't, because I went nuts with the documentation
also, I don't get much questions, period
(except for a few power users such as @aroberts , but these are way beyond "what implicit do I need?")
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:24
Yeah, when I used Kantan I just read the docs, had no questions.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:25
but I bet if we headed to the cats gitter, or the scalaz one, we'd see tons
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:25
If you documented more like me you'd have far more interactive users :D
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:25
I used to hate writing documentation. Then I discovered tut
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:26
I have an open feature I haven't made time to do, which answers all questions about imports with a specific answer, then I could add it to cats. Right now they don't want specific imports due to the way they structure things, so whatever import you need the answer is always cats.implicits. So I need to scan and find all enrichments but when it comes time to suggest what to import, I need to always answer cats.implicits.
Super easy feature, just so many fun things to work on I haven't made time to do it.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:28
oh, right, that'd be a problem for cats. Although I assume you could add meta-information to the generated json, such as consider cats.implicits to be the "bottom" import ?
then you work like the type inferrer. If you find anything more specific, use that. Otherwise go for Unit
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:29
Yep.
It's weird though, scala community doesn't have much love for clippy. RustClippy is huge, everyone uses it, over 1000 stars on github.
But at my work it really makes things easier for people, especially those new to Scala. I even catch little things, they type forEach and it fails, clippy reminds them it's foreach all lowercase.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:30
I'd never even heard of clippy before you brought it up
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:30
Oh.
I kind of assumed it was known, just not well loved.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:31
but I have a team of node developers to train to Scala and I'm definitely going to give this a serious look
it's honestly completely unknown in my corner of the scala community
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:31
Ask me any questions you have about it, it's easy enough to extend, and I am always in the clippy room.
Any question I get more than 2 or three times that involve me reading the compiler message and interpreting it for another dev, gets a clippy advice.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:32
we use a set of autoplugins to make SBT sane. Can I do something as simple as enable clippy through this so that we don't have to do any per-project manual configuration?
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:33
Newbies tend to take a chunk of code from one file, paste it into another then be lost when it doesn't compile, it's always imports. But which one?
Yep.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:33
interesting. I definitely need to take a look
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:33
And if you make a utility library with enrichments, if you add imclipitly to it the enrichments will be automatically available to anyone with clippy. To those without it, it does nothing so no harm done.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:34
how bad does it clash with sbt-errors ?
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:35
Never tried it.
Worth investigating. I have been wanting to try out sbt-errors but never made the time.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:35
I'm guessing pretty bad, since they both rework error messages
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:35
I would have guessed the opposite, but it's possible. They both append error messages, but I don't think either removes anything.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:36
sbt-errors does
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:36
Clippy never hides the original message, just adds more below it.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:36
it's actually a problem, it breaks lots of downstream things - such as sbt-emacs, for instance
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:36
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:36
yeah
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:37
I'll put it on my list to try out next weekend and if there are issues I'm sure I can work with Martin to resolve them.
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:38
one of the problems is that if you look at the example, you don't have the full path to files in which errors occur
might not be a problem for clippy though
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:39
No, clippy just regexes against the error messages. It's pretty simplistic, but still super useful. I mean 90% of the errors it helps with are of the form method [X] is not a member of type [Y] so you just catch that and tell the user to import com.my.yOps
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:40
I'll definitely have a look. I'm in the middle of coming up with a set of tools to use with my teams (hence the scalafmt questions), and this sounds like it might help me quite a bit by limitting the number of questions that reach me
Shane Delmore
@ShaneDelmore
Jul 08 2017 20:40
If you go to https://scala-clippy.org/ and click browse you can get a quick idea of some of the types of situations it is useful in.
They guide you towards putting advice in their online db, but I have found that much advice is either specific to your company, in which case you don't want it online, or specific to a version of a library (after all xor advice makes no sense in cats 0.9) so I added the ability to load advice from resources.
Off to find lunch. It's been fun chatting with you, thanks for the great idea on the animated Gif for the readme!
Nicolas Rinaudo
@nrinaudo
Jul 08 2017 20:43
my pleasure! Thanks for letting me know how you're using kantan.csv, that was really cool to learn
and I'll let you know if I have any question / issue with clippy!