nrinaudo/kantan.xpath

XPath wrapper for Scala

Howy Perrin

@howyp

haha! You’re welcome ;-)

Philippe Derome

@phderome

hello, saw some SO entry yesterday about kantan.xpath.

Nicolas Rinaudo

@nrinaudo

Probably authored by me, to be honest. It’s not exactly a popular library

Xml isn’t a thing that happens much in Scala world

Philippe Derome

@phderome

Yes indeed, this might be relevant to me someday as we use some XML at work

Nicolas Rinaudo

@nrinaudo

well, feel free to ask questions you might have about the project here, that's what it's made for. I try to keep the documentation comprehensive and up to date, so I encourage you to have a look if you'd like to learn more and see if it fits your needs

(and to have a look at its sister projects, kantan.regex and kantan.csv, for basically the same library, but for regular expressions and csv)

Philippe Derome

@phderome

hey I am interested in doing somewhat user-friendly XML diff of two documents programmatically. I found a number of useful references on the Java side and before I use them, I am curious as to whether some or much of it is already available in kantan.xpath. The input XML documents are created with a PrettyPrinter and thus have newlines in reasonable places. So the input to diff is pretty-printed XML docs (an assumption I can make that one would normally not make). I am looking at setting client data of line number with each node in the 2 XML documents I am comparing (I found this: https://eyalsch.wordpress.com/2010/11/30/xml-dom-2/). And finally XMLUnit for Java 2.x seems to have sufficient flexibility for what I am trying to do, namely order of child nodes is not important for comparison (something called DifferenceEvaluators.downgradeDifferencesToEqual(ComparisonType.CHILD_NODELIST_SEQUENCE)))

The general gist of what I am trying to do is something similar to Unix diff but accounting for some of the above requirements, showing what is different on both sides, what exists only one side and the line numbers associated with the nodes that appear in the report.

Nicolas Rinaudo

@nrinaudo

Honestly, as much as I’d like to advocate my lib here, it does nothing that can help you

Philippe Derome

@phderome

that is all right.

Nicolas Rinaudo

@nrinaudo

Kantan.xpath only lets you evaluate xpath expressions against an xml document

Philippe Derome

@phderome

Nicolas Rinaudo

@nrinaudo

The point being to get rid of the xml and turn it into useful typed as soon as possible, which is preciseky what you want not to do

Philippe Derome

@phderome

ha ha!

Nicolas Rinaudo

@nrinaudo

(Sorry about the typos, typing on a phone with a baby in one arm)

Philippe Derome

@phderome

no worries, and if we'd be speaking French here, you would not see the proper accents as I am on a US keyboard at work and cannot bother to make it multilingual

Stefan Endrullis

@xylo

Hi. I just searched for an xpath scala library again and stumbled over your project. And wow I'm really surprised! This project is fucking genius! - The compile time errors for xpath expressions, the automatic and intelligent type conversions, how trivial it is to define implicit decoders, ... This is just great and all I ever wanted! Thanks a lot!

Is there actually an overview over all kantan libs/projects?

Nicolas Rinaudo

@nrinaudo

Great to hear this is useful!

No overview yet, but the only directly useful ones are csv, xpath and regex

I’ve not updated them much recently because they’re pretty much done, but they could use a bit of love... just need to motivate myself

Stefan Endrullis

@xylo

It's definitely useful. Over the last years I tried several different libraries to query XML files. Some provide static type checking but have a cumbersome syntax and some use the original xpath syntax but do not provide compile time errors or nice type conversions. Yours is as far as I know the only one that combines both, and in a very elegant way.

OK, if it's so far only csv, xpath, and regex, then I can also find them via your github page. I thought there are also projects for kantan.codecs and kantan.text because I've read those names in the suggested conversations. But OK.

Nicolas Rinaudo

@nrinaudo

Kantan.text is kind of dead - it turned out not to be useful or fun. Kantan.codecs is the core encoding / decoding engine, it’s a “support” project. Kantan.sbt is another support project. Kantan.mongodb is... probably never going to be finished

Stefan Endrullis

@xylo

In case that someone uses Intellij IDEA: I just added support for kantan.xpath to my plugin called Custom Postfix Templates. It allows you to easily expand

"//my/query".xp

xp"//my/query"

by automatically adding the following imports:

import kantan.xpath._
import kantan.xpath.implicits._

Please let me know if I can improve these postfix templates.

Jisoo Park

@guersam

Thanks for making this library! How can I decode an optional field in a case class more tolerantly, like allowing an attribute having an empty string to be None?

Currently I'm adding a condition like [not(.='')] to the xpath query and it just works, but I wonder if there's more idiomatic way.

Nicolas Rinaudo

@nrinaudo

That seems pretty good to me - your xpath describes the shape of your data, and you use decoders to interpret it

You could also write a custon Decoder[String] that turns Some(“”) into Nonr

None

Jisoo Park

@guersam

Thanks for the quick response, I'll stick with this approach. Before that, I was trying to write a custom NodeDecoder[Option[A]] with Decoder.optionalDecoder and Decoder#handleErrorWith but failed to prove that there is an implicit IsError[DecodeError]. Do you have any clue?

Nicolas Rinaudo

@nrinaudo

Not without your code, no. Or did you mean a clue on how to write the decoder?

Nicolas Rinaudo

@nrinaudo

If you meant with the decoder, I think something like this should work:

NodeDecoder.optionNodeDecoder[String].filter(_ != "")

I would prefer to write:

implicit val lenientStringDecoder: NodeDecoder[Option[String]] = NodeDecoder[Option[String]].map(_.filter(_ != ""))

but that's not possible, Scala makes it unpleasant to "refine" an implicit value - given an implicit T, replace it with a new one adapted from it

(none of this code has been tried in the REPL, so if it doesn't compile, it's likely not your fault)

Jisoo Park

@guersam

I lost my first failed implementation and it's my second attempt without optionalDecoder:

  implicit def tolerantOptionDecoder[A](implicit ev: NodeDecoder[A]): NodeDecoder[Option[A]] =
    NodeDecoder.from {
      case Some(n) if n.getTextContent == "" => Right(None)
      case n @ Some(_) => ev.decode(n).map(Some(_))
      case None => Right(None)
    }

I'm not sure if Node#getTextContent makes sense here

Nicolas Rinaudo

@nrinaudo

Mmm... I might have misunderstood your intent

do you want a NodeDecoder[Option[String]] that's treats the empty string as a None, or do you want something that treats the empty string as the absence of node?

if the later, I'd need to play with it a bit, but I think a better solution would be to contramap on your decoder - turn Some(EmptyNode) into Node before the decoder is even called

Jisoo Park

@guersam

The former. I think contramap would work in this case too

Nicolas Rinaudo

@nrinaudo

if You want a NodeDecoder[Option[String]], then what I pasted before should work:

implicit val decoder: NodeDecoder[Option[String]] = NodeDecoder.optionNodeDecoder[String].filter(_ != "")

Mmm.... apparently it does not. Let me play with this for a bit

Nicolas Rinaudo

@nrinaudo

Right, this works:

def lenient[A: NodeDecoder](nonEmpty: A => Boolean) = NodeDecoder[Option[A]].map(_.filter(nonEmpty))

implicit val lenientString: NodeDecoder[Option[String]] = lenient(_ != "")

"<root id=''/>".evalXPath[Option[String]](xp"/root/@id")
// res1: kantan.xpath.XPathResult[Option[String]] = Right(None)

"<root id='foo'/>".evalXPath[Option[String]](xp"/root/@id")
// res2: kantan.xpath.XPathResult[Option[String]] = Right(Some(foo))

 "<root/>".evalXPath[Option[String]](xp"/root/@id")
// res3: kantan.xpath.XPathResult[Option[String]] = Right(None)

There. Was that what you had in mind?

Owen Healy

@ellbur

so, this is probably a silly question, but if I want to run multiple queries on the same document, is there a way to parse it once and run .evalXPath repeatedly on the result?

Nicolas Rinaudo

@nrinaudo

Sure, you can take any node org.w3c.dom.Node and execute all the xpath evaluation methods on it

this is achieved through the XmlSource type class, which provides the instance you need: https://github.com/nrinaudo/kantan.xpath/blob/master/core/src/main/scala/kantan/xpath/XmlSource.scala#L101

if you need to store your documents in another type, no worries, just declare an implicit XmlSource[YourDocumentType]

Where communities thrive

People

Repo info

Activity