These are chat archives for thunder-project/thunder

3rd
Mar 2015
tomsains
@tomsains
Mar 03 2015 10:24
Hello,
tomsains
@tomsains
Mar 03 2015 10:30
I have been finding the new meanOfRegions function really useful - a really great addition to thunder! One thing I have been having trouble with is exporting the resulting series. When I attempt to save the series using the export function it returns a error stating the array size must remain unchanged. What conversion should I be doing so that I can export successfully?
tomsains
@tomsains
Mar 03 2015 12:36
*meanByRegions
Jason Wittenbach
@jwittenbach
Mar 03 2015 14:12
@tomsains Are you using the trying to write the resulting Images object directly with something like image.meanByRegions().exportAsPngs() ?
If so, it might be better to try converting to a Series first with something like image.meanByRegions().toSeries().saveAsBinarySeries()
tomsains
@tomsains
Mar 03 2015 14:25
ah @jwittenbach I had been calling it on a series object not on a Images object. I have been using the export() not saveAsBinarySeries() function because I wanted to be able to save it as a text or MAT file so that I could plot their timeseries locally (since seaborn isn't compatible with the version of python installed on the cluster).
Jason Wittenbach
@jwittenbach
Mar 03 2015 15:11
Ah, ok, that makes sense. After you convert the output of meanByRegions to a Series, does something simpler like calling series.pack() work?
Jeremy Freeman
@freeman-lab
Mar 03 2015 15:47
@tomsains glad you're finding that useful! Just to clarify for everyone, if you call meanByRegions on a Series, you get back another Series, let's call it data, where every record is the time series from one region. If you want to just grab that whole thing as an array (i.e. the full set of time series), you'll want to use the data.collectValuesAsArray() method. That will give you a local array with the time series, and is reasonable to do under the assumption that the number of time series isn't too terribly large.
You could also call k, v = data.collectAsArray() which will give you two local arrays, one with the centers of the regions and another with the time series.
And then you can finally call export on those arrays to save to mat, npy, or txt
tomsains
@tomsains
Mar 03 2015 16:18
Ah brilliant, that makes sense. Have collectedAsArray() and can now export. Thank you once again.
Jeremy Freeman
@freeman-lab
Mar 03 2015 17:17
Great! One more note: after this PR (thunder-project/thunder#124) export will be exposed as a method on the ThunderContext, so you'll call tsc.export instead of importing a separate function. Among other things, this new version allows you to write data to S3 as well as to a local file system on the cluster, which might be useful for more permanent storage.
If you update to master branch you'll have this change now, and it will be included in the 0.5.0 release
tomsains
@tomsains
Mar 03 2015 17:48
Yeah thank you. I have this change now, managed to figure out the API change - definitely finding it useful being able to export to S3 directly!