geotrellis/geotrellis

GeoTrellis is a geographic data processing engine for high performance applications.

Grigory

@pomadchin

will create an issue with details

iceland1906

@iceland1906

Nice! thanks for sorting this out! @pomadchin

Eugene Cheipesh

@echeipesh

@metasim that's not set in stone. If you think it would really simplify cliet code I'd be game to give it another shot for next release but basically we were in "this is not making things better for the few days we have to spend at this" category.

Grigory

@pomadchin

@iceland1906 suprsigingly it looks like it never worked

@echeipesh @metasim I’m totally fine with having both envelope and extents everywhere with extent being more restrcited but being simple and immutable; PostGIS also has two separate functions where ST_extent works with bbox and ST_envelope works with aribtrary geometries and can even work with lines / points

Simeon H.K. Fitch

@metasim

Mutability: say no more

Eugene Cheipesh

@echeipesh

Maybe we can get an immutable envelope into next JTS version? That seems ideal.

Grigory

@pomadchin

:+1: // sounds like +1000

Simeon H.K. Fitch

@metasim

@jnh5y ?

+1e50

For a few days, you guys killed it!

James Hughes

@jnh5y

For these immutable envelopes, will they be sealed already? If so, how do you expect to get the birthday card inside?;)

I like the idea; ask in the JTS channel.

Simeon H.K. Fitch

@metasim

In functional programming, the birthday card doesn't exist until you open the envelope.

James Hughes

@jnh5y

I don't know if we'll be in a situation where we'd need 'Envelope' to be an interface, etc, etc

Grigory

@pomadchin

oh no, a function that will create a copy of an envelope with a card inside

Simeon H.K. Fitch

@metasim

of course... silly me

James Hughes

@jnh5y

how does structural sharing interact with Platonic ideals?

Simeon H.K. Fitch

@metasim

There's a Platonic notion of sharing, encompassing the set of all possible relations

@pomadchin Did you guys run into this error when developing GT 3.0?

com.fasterxml.jackson.databind.JsonMappingException: Incompatible Jackson version: 2.9.8

Grigory

@pomadchin

@metasim sure

Simeon H.K. Fitch

@metasim

Oddly, it doesn't show up in the evicted report.

Grigory

@pomadchin

@metasim https://github.com/locationtech/geotrellis/blob/master/project/Settings.scala#L315-L327

Simeon H.K. Fitch

@metasim

:win:

Grigory

@pomadchin

it is because of a new aws sdk :/

Simeon H.K. Fitch

@metasim

:sadness:

Grigory

@pomadchin

@iceland1906 locationtech/geotrellis#3133

iceland1906

@iceland1906

:+1:

Simeon H.K. Fitch

@metasim

This is kinda a Scala typing question, but I'm confused about why these aren't both true (and it's causing Spark Catalyst issues):

scala> classOf[geotrellis.raster.CellGrid[_]].isAssignableFrom(classOf[geotrellis.raster.Tile])
res0: Boolean = false

scala> classOf[geotrellis.raster.CellGrid[_]].isInstance(geotrellis.raster.IntConstantTile(1,2,3))
res1: Boolean = true

As a refresher, the docs on isAssignableFrom say:

Determines if the class or interface represented by this Class object is either the same as, or is a superclass or superinterface of, the class or interface represented by the specified Class parameter.

This is also false:

scala> classOf[geotrellis.raster.CellGrid[Int]].isAssignableFrom(classOf[geotrellis.raster.Tile])
res2: Boolean = false

This is only now relevant for me due to CellGrid now having a type parameter. Thoughts? Something obvious?

Eugene Cheipesh

@echeipesh

Not obvious to me, checking it out

classOf wouldn't hold any information about the type parameter though

Simeon H.K. Fitch

@metasim

I just noticed that CellGrid is an abstract class, and Tile extents it... didn't even know that was possible!

Ok, this is super spooky.... CellGrid doesn't show up through reflection!:

scala> classOf[Tile].getInterfaces
scala> classOf[Tile].getSuperclass
scala> classOf[Tile].getGenericInterfaces

res0: Array[Class[_]] = Array(interface geotrellis.raster.IterableTile, interface geotrellis.raster.MappableTile)
res1: Class[_ >: geotrellis.raster.Tile] = null
res2: Array[java.lang.reflect.Type] = Array(interface geotrellis.raster.IterableTile, geotrellis.raster.MappableTile<geotrellis.raster.Tile>)

Simeon H.K. Fitch

@metasim

Maybe it's some kind of Scala oddity:

abstract class Foo {
  val blah: String = "foo"
}

trait Bar extends Foo {
  val merf: String = "bar"
}

classOf[Foo].isAssignableFrom(classOf[Bar])
...
res3: Boolean = false

Simeon H.K. Fitch

@metasim

@echeipesh I think this is some fundamental aspect of Scala I didn't know existed. Taking the question to Scala Users.

Grigory

@pomadchin

@metasim I think it is similar to mixins restrictions behavior

Grigory

@pomadchin

@metasim https://alvinalexander.com/scala/scala-trait-examples#limiting-which-classes-can-use-a-trait

Simeon H.K. Fitch

@metasim

Not sure this is my issue....
The issue is that if one uses reflection to instantiate a class with ctor parameters (t: CellGrid[Int], e: Extent) and pass it (t: Tile, e: Extent), it will fail.

Grigory

@pomadchin

ah I was talking about how traits can extend classes

even not abstract

damn sounds like a bug in our hierarchy

Simeon H.K. Fitch

@metasim

Yeh, I'm more hung up on the fact that the relationship doesn't exist in the JVM. Here's the trigger:

scala> val rc = classOf[Raster[Tile]]
rc: Class[geotrellis.raster.Raster[geotrellis.raster.Tile]] = class geotrellis.raster.Raster

scala> rc.getConstructors
res0: Array[java.lang.reflect.Constructor[_]] = Array(public geotrellis.raster.Raster(geotrellis.raster.CellGrid,geotrellis.vector.Extent))

scala> rc.getConstructor(classOf[Tile], classOf[Extent])
java.lang.NoSuchMethodException: geotrellis.raster.Raster.<init>(geotrellis.raster.Tile, geotrellis.vector.Extent)

However, it's clearly in the compile-time typing:

scala> import scala.reflect.runtime.universe._
scala> typeOf[Tile].baseClasses

res0: List[reflect.runtime.universe.Symbol] = List(trait Tile, trait MappableTile, trait MacroMappableTile, trait IterableTile, trait MacroIterableTile, class CellGrid, class Grid, trait Serializable, trait Serializable, class Object, class Any)

Grigory

@pomadchin

yea I see

Simeon H.K. Fitch

@metasim

The reason this is relevant for me has to do with Spark SQL's encoding of product types (e.g. case classes) expects a ctor to exist:

scala> rf.printSchema()
raster
 |-- tile: tile (nullable = true)
 |-- extent: struct (nullable = true)
 |    |-- xmin: double (nullable = false)
 |    |-- ymin: double (nullable = false)
 |    |-- xmax: double (nullable = false)
 |    |-- ymax: double (nullable = false)

scala> rf .select(col("raster").as[Raster[Tile]])

ERROR CodeGenerator: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 102, Column 11: No applicable constructor/method found for actual parameters "geotrellis.raster.Tile, geotrellis.vector.Extent"; candidates are: "geotrellis.raster.Raster(geotrellis.raster.CellGrid, geotrellis.vector.Extent)"
org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 102, Column 11: No applicable constructor/method found for actual parameters "geotrellis.raster.Tile, geotrellis.vector.Extent"; candidates are: "geotrellis.raster.Raster(geotrellis.raster.CellGrid, geotrellis.vector.Extent)"

Grigory

@pomadchin

yea I got it why you need it :/

Simeon H.K. Fitch

@metasim

Not really a bug in your hierarchy.

It's that Spark SQL Encoders make a lot of assumptions.

Grigory

@pomadchin

yea Sparky reflection

@metasim

:javap geotrellis.raster.Tile
  Size 5432 bytes
  MD5 checksum 130428f0bb0a80b0748ad561f426f1ab
  Compiled from "Tile.scala"
public interface geotrellis.raster.Tile extends geotrellis.raster.IterableTile, geotrellis.raster.MappableTile<geotrellis.raster.Tile>

will you file a GT issue? sounds like smth we need to resolve anyway

Where communities thrive

People

Repo info

Activity