In my re-implementation of the Gaia Sky level-of-detail (LOD) catalog generation in Rust I have been able to roughly halve the processing time, and, even though I do not have concrete numbers yet, everything points towards a drastic decrease in memory usage as well. In this project, I need to read a metric ton of gzipped csv Gaia catalog files, parse and process them into a functional in-memory catalog with cartesian positions, velocity vectors, RGB colors, etc. Then I need use them to generate an octree that represents the LOD structure, and finally write another metric ton of binary files back to disk. Using memory mapped files helps a lot in avoiding copies and speeding up the reading and writing operations; that’s something I tried out in the Java version and have come to also re-implement in Rust. Here’s the thing though: working with memory mapped files in Java is super straightforward. In Rust? Not so much. And the lack of available documentation and examples does not help. I was actually unable to find any working snippets with all the parts I needed, so I’m documenting it in this post in case someone else is in the same situation I was.

A little over a year ago, in January 2020, I got myself a QNAP TS-351-2G 3-bay NAS in order to store all of my and my family’s data in a fail safe RAID configuration. I opted for the somewhat unconventional 3-bay setup in an attempt to trade off limited physical space at home with storage capacity. I don’t have much space in my living room for a big NAS, and the 2-bay options, albeit being very compact, are limited to RAID-1, where half of the space is used for storage and the other half is used for redundancy protection (data is basically mirrored on the second drive). In QNAP’s website there are three 3-bay Home options: the entry-level TS-332X, the middle-range TS-328 and the high-end TS-351. So I thought to myself, “I’m getting the high-end unit, how bad can it be?”. Well, now that I have been using this NAS for a year I think I can answer this question.

The other day I was browsing through the Wayback Machine and I tried looking up if they had a copy of my old blog dimlight.wordpress.com, a blog I wrote during my second Erasmus exchange program stay in the University of Aberdeen, Scotland. Of course, they had several snapshots archived. The images and videos are missing, but the texts are intact, so I decided to re-publish verbatim copies of these old posts here.

If you usually develop your software without an IDE, it may come in handy to be able to run a custom command or two whenever a file or a group of files in the file system is modified. This post discusses entr, a small event notify test runner which might just be what you need to fill an inconvenient gap in your mouseless development environment.

Lately I’ve taken an interest in the Rust programming language. I read the epub version of the Rust book over summer and found it quite well-written and overall interesting. Recently, I got myself a hard copy that sits on my desktop for quick queries, as I’ve been re-implementing the Gaia Sky catalog generation in Rust. The reasons for this are varied, but basically they come down to hardware. We need to load a humongous amount of stars (~1.6 billion) into memory, our old server, which had 2 TB of RAM, is being replaced by a newer, faster, and smaller (memory-wise, ~700 GB) machine in which we won’t be able to generate with the current Java implementation. But this is a story for another post. Today I’m here to document the completely mouseless Rust development environment I’ve been putting together as a sort of side project, to aid in my re-implementation.

At home, I have a scrawny HTPC called chimp in my living room connected to the TV —as I don’t own a Smart TV for good reasons—. Even though I have a NAS in the network capable of serving media, I connected a dedicated external disk directly to chimp because my stock router is not the fastest around. Whenever I use the HTPC, I use it remotely from either my desktop, bonobo, or my laptop, simian. Sometimes I need to fetch torrents and download them to the disk connected to the HTPC. Enter Transmission. Transmission is a somewhat popular BitTorrent client that includes a ‘hidden’ command line interface which is very, very useful and simple to use. Learn to use it and you will probably never want to open a GUI torrent client ever again.