geotrellis/geotrellis

hello ;)

i am having an issue with a union over various singleband layers
the groupByKey function is very, very slow (even says so in the documentatation that is is very heavy)
any ideas on approaches on how i can get rid of the groupByKey after the union. what i need to get is somekind of an iterable on the singleband tiles - that's why i am doing the groupByKey, but i need another approach....

//Case class extending a tile with an Int (band index) so that we can identify where a specific tile belongs to
case class TileWithBandIndex(tile: Tile, bandIndex: Int)

//Read input layers
val inputLayers = Array.ofDim[RDD[(SpatialKey, TileWithBandIndex)]](2)
inputLayers(0) = reader.read[SpatialKey, Tile, TileLayerMetadata[SpatialKey]](LayerId("Norway", 0)).map {
          case (key, tile) => (key, TileWithBandIndex(tile, 0))
        }
inputLayers(1) = reader.read[SpatialKey, Tile, TileLayerMetadata[SpatialKey]](LayerId("Austria", 0)).map {
          case (key, tile) => (key, TileWithBandIndex(tile, 1))
        }

//Combine (union) the input layers and perform some logic
LOG.info(s"Combine input layers and perform some")
val extractedLayer =
  sc.union(inputLayers)
    .groupByKey()
    .map{
          case (key, tilesWithBandIndex) => {
            //Create an array of tiles based on the tiles available at this key. Fill missing bands with empty tiles
            val tiles = Array.ofDim[Tile](inputLayers.size)
            var refTile: Tile = null
            tilesWithBandIndex.foreach(element => {
              tiles(element.bandIndex) = element.tile
              if (refTile == null) {refTile = element.tile}
            })
            for (bandIndex <- 0 until inputLayers.size) {
              if (tiles(bandIndex) == null) {
                tiles(bandIndex) = ArrayTile.empty(refTile.cellType, refTile.cols, refTile.rows)
              }
            }
            //Create a multiband tile from the tiles at the given key and perform the extration logic on it
            (key, myVeryOwnMapTileFunctionThatDoesSomethingWithAMultibandTileAndReturnsASinglebandTile(MultibandTile(tiles)))
          }
        }
    })

i did some further analysis over the weekend
it turned out, it is not the union nor the groupbykey() that is making the "union" slow, but the amount of data needed from S3 to create the union.
in my case we are talking about 11k tiles per layer (512x512) and 48 layers are being combined in the union. So there is hardly any CPU load nor network traffic during the union process as I guess most of the overhead comes from building up new connections. One tile size is ~300bytes.

Increasing the tile size significantly increases the performance but at the cost of very, very large RAM consumption. Of course. ;)

So I am wondering how other are dealing the the issue of serving tiles from an S3 storage.
More spark nodes and going more in parallel do make the union (download) step faster?