Wednesday, August 12, 2015

Mosstack 0.7 development and problems

After Mosstack 0.6 which I realeased last year, I've been writing a new command line interface for it and that development has been quite slow. Coding math related stuff is nice, but I don't like interfaces. That added to the fact that stargazing and photographing weathers were extremely rare during last winter made me concentrate on other things. First I thought I'd get more programming done with no photos to process, but the lack of photos seemed to make the programming quite pointless. I also started another open source project with SailFish OS, a gym logbook for Jolla phones...

Astropy got updated to 1.0 and that broke my program. I started getting error while reading a Fits file the program wrote itself:

ValueError: Cannot load a memory-mapped image: BZERO/BSCALE/BLANK header keywords present. Set memmap=False.

It took me a while to understand and fix. The files are quite big and there might be a lot of them in the memory simultaneously so memmap is an important feature and can't be set False. Internet wasn't helpful at first. I tried removing said keywords from header, but it didn't help. Finally I found something that said Fits format not supporting datatype uint16. That was what I was using. I changed it to int32 and everything works.

Another problem was a memory leak with my new C extension VNG debayering method. Each debayer took about 300 MiB of memory and that was never released. I hunted that leak for days with memory_profiler and finally located it to the C extension. Even then I didn't see it myself but someone else pointed out I allocating memory inside a loop but freeing it only after the loop. Perhaps I really should learn to code C before attempting anything with it...

The new CLI is perhaps half done. You can run a simple stacking process with it but many features are still to be implemented. My plan is to get 0.7 done before the stargazing season in Finland begins. That means about a month.

Planned features:
  • New CLI
  • VNG debayering with C extension and C binary debayer (done)
  • Decoding DSLR raw photo to FITS with a C extension so decoding would work without dcraw and imagemagick (almost done)
  • New GUI with QML (moved to next release)