freeman-lab on master
Update README.md (compare)
freeman-lab on master
add link back (compare)
freeman-lab on master
remove links (compare)
jwittenbach on master
sets split to correct value aft… Merge pull request #108 from jw… (compare)
data = thunder.images.fromtifwithmetadata(row['filepath'], engine=sc, labels=(json.dumps(row),))
rdd = data.tordd().map(lambda ((a,b),c): ((a,b),bz2.compress(pickle.dumps(c))))
rdd.saveAsPickleFile("/data/savedFromThunder")
rdd = sc.pickleFile("/data/savedFromThunder")
data = thunder.images.fromrdd(rdd, nrecords=None, dims=(900,900), dtype='string')
data.toarray()
import thunder as td; import numpy as np
rdd = sc.parallelize([( (0,), np.random.randn(900, 900) ), ( (1,), np.random.randn(900, 900) )])
imgs = td.images.fromrdd(rdd, dims=(900, 900))
a = imgs.toarray()
rdd.first()[1]
load_pickle
that takes a filename and returns a NumPy ndarray
containing the data from the loaded file
glob
to generate a list containing all of the file names
dtype
argument based on the dtype that the arrays from your load_pickle
function have
len(fnames)
will return the total # of such arrays
number x height x width
, which is what fullshape
will be