These are chat archives for astropy/astropy
bsipocz@ddasilva - As Cadair said above, it's not totally clear what you would like to know, so my response may not be the answer you're looking for:
wcsmodules while remote ones are primarily in
visualization.wcsaxesand a few in
astropy.coordinates. Look for the
@remote_datadecorator. There are a few helpers utility in astropy.utils.data to help deal with test datasets.
Cadairok so can someone tell me if this is a feature or a bug
CadairI am reading a delerberaly broken FITS file
Cadair(I opened it in a HEX editor and broke it)
CadairI changed the NAXIS1 key to be the wrong number
Cadairboth smaller and larger
Cadairand io.fits will happily read the file anyway with out even a warning
astrofrogdid you make it large enough that it should read beyond the end of the file
astrofrogthen normally it should emit a truncation warning
CadairI would think
CadairI will make it silly large
astrofroglet me try here too
astrofrog In : from astropy.io import fits
In : hdu = fits.open('fake.fits')
WARNING: File may have been truncated: actual file length (1045440) is smaller than the expected size (10388160) [astropy.io.fits.file]
CadairI have seen that before
Cadairso why was it not emitting it
Cadairhow did you break it?
astrofrogI just made NAXIS1 10 times bigger
astrofrogcan you share the file you are using?
Cadairok I only made it slightly wrong
astrofrogthere is sometimes padding
astrofrogso you have to make it quite wrong
Cadairok that's annoying
CadairI want it to tell me if it's any level of wrong lol
astrofrogyeah in the hex editor look at the end of the file, there will be a bunch of 0000
astrofrogdoes does it warn if you make it very wrong at least?
astrofrogFITS is made up of 2880 byte data blocks
astrofrogyeah all my FITS files have a size that is a multple of 2880 bytes
astrofrogwhat a waste of space
Cadairwhat about the other way around
Cadairwhat if you make the header smaller than the data?
astrofrogif there is only one HDU it won't matter
astrofrogif there are multiple HDUs it probably will barf if you try and access HDUs > 0
astrofrogBUT we could potentially check if there is non-zero data in the remainder of the last 2880 byte block that is read
astrofrogas that would indicate an issue
astrofrogio.fits doesn't do that but it could
astrofroghowever not sure if there is a requirement padding has to be zeros
astrofrogFITS says '
astrofrogso that is one thing that could be checked
astrofrogbut isn't currently
astrofrogmaybe could open an issue or PR for that?
Cadairthis is actual work so why not
astrofrogI mean you are screwed if the size you are reading is a multiple of 2880 because you shouldn't continue to read the next one
astrofrogbut at least in the case where product of naxis is not multiple of 2880, can check when reading last block
Cadairok so that's the next thing I am going to try and check
Cadairwhat happens if I change the amount of actual data
astrofrogin what sense?
astrofrogI mean if you add fake data at end of FITS file?
astrofrogor if you truncate file without changing naxis?
astrofrogI think the same as what would happen when changing naxis
astrofrogwell for a start if you added data that wasn't a multiple of 2880 bytes that's easy to catch
astrofrogso that's something that could be checked, I think all FITS files should be a multiple of 2880
astrofrogif you add n x 2880 bytes they will just be ignored
astrofrogbut that should also raise a warning because after the 2880 byte blocks for the data it should either stop or start a new header
astrofrogso if it's just more data bocks can warn
astrofrogand if you remove n x 2880 blocks it should emit a warning as it does at the moment
astrofrogso I think you can catch most invalid cases actually, it just doesnt' do it right now
astrofrogThe lowest hanging fruit I think is probably checking that the file size is multiple of 2880 since if it's not that's a red flag
astrofrogthen the next thing to check is that when reading the last 2880 bytes of a data record, ignored bytes are all zero
astrofrogasdf is much nicer in this respect...
CadairI can't really see a reason for it
Cadairis it just so you can chunk the IO?
astrofrogprobably 'historical reasons'
astrofrogheaders have padding too - I think that's so that you can edit the header without re-writing the file