geotrellis/geotrellis

GeoTrellis is a geographic data processing engine for high performance applications.

Simeon H.K. Fitch

@metasim

Mutability: say no more

Eugene Cheipesh

@echeipesh

Maybe we can get an immutable envelope into next JTS version? That seems ideal.

Grigory

@pomadchin

:+1: // sounds like +1000

Simeon H.K. Fitch

@metasim

@jnh5y ?

+1e50

For a few days, you guys killed it!

James Hughes

@jnh5y

For these immutable envelopes, will they be sealed already? If so, how do you expect to get the birthday card inside?;)

I like the idea; ask in the JTS channel.

Simeon H.K. Fitch

@metasim

In functional programming, the birthday card doesn't exist until you open the envelope.

James Hughes

@jnh5y

I don't know if we'll be in a situation where we'd need 'Envelope' to be an interface, etc, etc

Grigory

@pomadchin

oh no, a function that will create a copy of an envelope with a card inside

Simeon H.K. Fitch

@metasim

of course... silly me

James Hughes

@jnh5y

how does structural sharing interact with Platonic ideals?

Simeon H.K. Fitch

@metasim

There's a Platonic notion of sharing, encompassing the set of all possible relations

@pomadchin Did you guys run into this error when developing GT 3.0?

com.fasterxml.jackson.databind.JsonMappingException: Incompatible Jackson version: 2.9.8

Grigory

@pomadchin

@metasim sure

Simeon H.K. Fitch

@metasim

Oddly, it doesn't show up in the evicted report.

Grigory

@pomadchin

@metasim https://github.com/locationtech/geotrellis/blob/master/project/Settings.scala#L315-L327

Simeon H.K. Fitch

@metasim

:win:

Grigory

@pomadchin

it is because of a new aws sdk :/

Simeon H.K. Fitch

@metasim

:sadness:

Grigory

@pomadchin

@iceland1906 locationtech/geotrellis#3133

iceland1906

@iceland1906

:+1:

Simeon H.K. Fitch

@metasim

This is kinda a Scala typing question, but I'm confused about why these aren't both true (and it's causing Spark Catalyst issues):

scala> classOf[geotrellis.raster.CellGrid[_]].isAssignableFrom(classOf[geotrellis.raster.Tile])
res0: Boolean = false

scala> classOf[geotrellis.raster.CellGrid[_]].isInstance(geotrellis.raster.IntConstantTile(1,2,3))
res1: Boolean = true

As a refresher, the docs on isAssignableFrom say:

Determines if the class or interface represented by this Class object is either the same as, or is a superclass or superinterface of, the class or interface represented by the specified Class parameter.

This is also false:

scala> classOf[geotrellis.raster.CellGrid[Int]].isAssignableFrom(classOf[geotrellis.raster.Tile])
res2: Boolean = false

This is only now relevant for me due to CellGrid now having a type parameter. Thoughts? Something obvious?

Eugene Cheipesh

@echeipesh

Not obvious to me, checking it out

classOf wouldn't hold any information about the type parameter though

Simeon H.K. Fitch

@metasim

I just noticed that CellGrid is an abstract class, and Tile extents it... didn't even know that was possible!

Ok, this is super spooky.... CellGrid doesn't show up through reflection!:

scala> classOf[Tile].getInterfaces
scala> classOf[Tile].getSuperclass
scala> classOf[Tile].getGenericInterfaces

res0: Array[Class[_]] = Array(interface geotrellis.raster.IterableTile, interface geotrellis.raster.MappableTile)
res1: Class[_ >: geotrellis.raster.Tile] = null
res2: Array[java.lang.reflect.Type] = Array(interface geotrellis.raster.IterableTile, geotrellis.raster.MappableTile<geotrellis.raster.Tile>)

Simeon H.K. Fitch

@metasim

Maybe it's some kind of Scala oddity:

abstract class Foo {
  val blah: String = "foo"
}

trait Bar extends Foo {
  val merf: String = "bar"
}

classOf[Foo].isAssignableFrom(classOf[Bar])
...
res3: Boolean = false

Simeon H.K. Fitch

@metasim

@echeipesh I think this is some fundamental aspect of Scala I didn't know existed. Taking the question to Scala Users.

Grigory

@pomadchin

@metasim I think it is similar to mixins restrictions behavior

Grigory

@pomadchin

@metasim https://alvinalexander.com/scala/scala-trait-examples#limiting-which-classes-can-use-a-trait

Simeon H.K. Fitch

@metasim

Not sure this is my issue....
The issue is that if one uses reflection to instantiate a class with ctor parameters (t: CellGrid[Int], e: Extent) and pass it (t: Tile, e: Extent), it will fail.

Grigory

@pomadchin

ah I was talking about how traits can extend classes

even not abstract

damn sounds like a bug in our hierarchy

Simeon H.K. Fitch

@metasim

Yeh, I'm more hung up on the fact that the relationship doesn't exist in the JVM. Here's the trigger:

scala> val rc = classOf[Raster[Tile]]
rc: Class[geotrellis.raster.Raster[geotrellis.raster.Tile]] = class geotrellis.raster.Raster

scala> rc.getConstructors
res0: Array[java.lang.reflect.Constructor[_]] = Array(public geotrellis.raster.Raster(geotrellis.raster.CellGrid,geotrellis.vector.Extent))

scala> rc.getConstructor(classOf[Tile], classOf[Extent])
java.lang.NoSuchMethodException: geotrellis.raster.Raster.<init>(geotrellis.raster.Tile, geotrellis.vector.Extent)

However, it's clearly in the compile-time typing:

scala> import scala.reflect.runtime.universe._
scala> typeOf[Tile].baseClasses

res0: List[reflect.runtime.universe.Symbol] = List(trait Tile, trait MappableTile, trait MacroMappableTile, trait IterableTile, trait MacroIterableTile, class CellGrid, class Grid, trait Serializable, trait Serializable, class Object, class Any)

Grigory

@pomadchin

yea I see

Simeon H.K. Fitch

@metasim

The reason this is relevant for me has to do with Spark SQL's encoding of product types (e.g. case classes) expects a ctor to exist:

scala> rf.printSchema()
raster
 |-- tile: tile (nullable = true)
 |-- extent: struct (nullable = true)
 |    |-- xmin: double (nullable = false)
 |    |-- ymin: double (nullable = false)
 |    |-- xmax: double (nullable = false)
 |    |-- ymax: double (nullable = false)

scala> rf .select(col("raster").as[Raster[Tile]])

ERROR CodeGenerator: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 102, Column 11: No applicable constructor/method found for actual parameters "geotrellis.raster.Tile, geotrellis.vector.Extent"; candidates are: "geotrellis.raster.Raster(geotrellis.raster.CellGrid, geotrellis.vector.Extent)"
org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 102, Column 11: No applicable constructor/method found for actual parameters "geotrellis.raster.Tile, geotrellis.vector.Extent"; candidates are: "geotrellis.raster.Raster(geotrellis.raster.CellGrid, geotrellis.vector.Extent)"

Grigory

@pomadchin

yea I got it why you need it :/

Simeon H.K. Fitch

@metasim

Not really a bug in your hierarchy.

It's that Spark SQL Encoders make a lot of assumptions.

Grigory

@pomadchin

yea Sparky reflection

@metasim

:javap geotrellis.raster.Tile
  Size 5432 bytes
  MD5 checksum 130428f0bb0a80b0748ad561f426f1ab
  Compiled from "Tile.scala"
public interface geotrellis.raster.Tile extends geotrellis.raster.IterableTile, geotrellis.raster.MappableTile<geotrellis.raster.Tile>

will you file a GT issue? sounds like smth we need to resolve anyway

Grigory

@pomadchin

and in 2.x:

:javap Tile
  Size 15801 bytes
  MD5 checksum 603f5495ea4c630a86414555994a052a
  Compiled from "Tile.scala"
public interface geotrellis.raster.Tile extends geotrellis.raster.CellGrid, geotrellis.raster.IterableTile, geotrellis.raster.MappableTile<geotrellis.raster.Tile>

15952026052

@15952026052

Hello When I upgraded to 3.0.0, the original guide package error occurred.
How can I fix it?

Grigory

@pomadchin

hey @15952026052

import geotrellis.store._
import geotrellis.store.index._
import geotrellis.store.file._
import geotrellis.store.hadoop._

Where communities thrive

People

Repo info

Activity